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PREFACE 


For  the  seventeenth  year,  the  Research  and  Theory  Division  of  the  Association  i'or 
Educational  Communications  and  Technology  (AECT)  is  sponsoring  the  pub.-  -.lions  of  these 
Proceedings.  Papers  published  in  this  volume  were  presented  at  the  national  AECT  Convention  in 
Anaheim,  CA.  A  limited  quantity  of  this  volume  were  printed  and  sold.  It  is  also  available  on 
microfiche  through  the  Educational  Resources  Information  Clearinghouse  (ERIC)  system. 

REFEREEING  PROCESS:  All  research  papers  selected  for  presentation  at  the  AECT 
convention  and  included  in  this  Proceedings  were  subjected  to  a  rigorous  blind  reviewing  process. 
All  references  to  author  were  removed  from  proposals  before  they  were  submitted  to  referees  for 
review.  Approximately  fifty  percent  of  the  manuscripts  submitted  for  consideration  were  selected 
for  presentation  at  the  Convention  and  for  publication  in  these  Proceedings.  The  papers  contained 
in  this  document  represent  some  of  the  most  current  thinking  in  educational  communications  and 
technology. 

A  selected  number  of  development  papers,  sponsored  by  the  Division  for  Instructional 
Development  (DID),  are  included  in  this  Proceedings.  The  most  important  instructional 
development  papers  were  selected  by  the  DID  program  chairs  for  publication. 

This  volume  contains  two  indexes.  The  first  is  an  author  index;  the  second  is  a  descriptor 
index.  The  index  for  volumes  1-6  (1979-84)  is  included  in  the  1986  Proceed.in^s,  and  the  index 
for  volumes  7-10  is  in  the  1988  Proceedings.  After  1988,  each  volume  contains  indexes  for  that 
year  only. 


M.  R.  Simonson 
Editor 


RESEARCH  AND  THEORY  DIVISION  OFFICERS 


President:  Gary  R.  Morrison 

Memphis  State  University 
972  Grouse  Meadow  Dr. 
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President-elect:  Patricia  Smith 

Department  of  Educational  Psychology 

University  of  Oklahoma 
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405-325-2882 
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e-mail  bwilson@cudnvr.denver.colorado.edu 


Publications  editor:  Kathryn  L.  Ley 

Grambling  State  University 
14  University  Blvd 
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JOIN  AECT  TODAY! 

AECT  hes  eleven  divisions  to  meet  your  special  internals.  Each  division  publishes  its  own  newsletter  and  most  divisions  sponsor  an 
awards  prog, am.  Your  AECT  membership  includes  one  division,  and  you  may  join  as  many  divisions  as  you  like  tor  $10  each. 


Division  tor  Systemic  Change  !n  Education  (CHANGE)  promoles 
syslomlc  chango  In  schools,  businesses,  higher  education,  and  technical 
institutes,  primarily  through  systems  design  approaches,  lo  ineel  learners' 
needs,  improve  the  quality  ol  education,  and  lor  enabling  technology  lo 
reach  its  potential  In  all  educational  settings 

Division  ot  Educational  Media  Management  (DEMM)  shams 
information  on  common  problems,  provides  solutions  and  progiam 
descriptions  ol  educational  media  management  programs  thai  help  carry 
out  media  management  responsibilities  elleclively. 

Division  ol  Instructional  Development  (DID)  studies,  evaluates,  and 
relinos  design  processes  including  analysis  techniques  and  consullalion 
strategies;  disseminalos  research  findings  pertinent  lo  these  processes; 
creates  new  models  ol  instructional  development;  and  promoles 
academic  programs  in  instructional  technology. 
Division  ol  Interactive  Systems  and  Computers  (DISC)  is  concerned  ■ 
with  the  generation,  access,  organisation,  storage,  and  delivery  ol  all 
lorms  ol  inlormation  used  in  the  processes  ol  education  and  training.  The 
division  promoles  the  networking  ol  ils  members  lo  lacilifate  sharing  ol 
expertise  and  interests. 

Division  of  Learning  and  Performance  Environments  (DUPE) 
supports  human  learning  and  performance  through  the  use  ol  compuler- 
based  technology:  design,  developmonl,  evaluation,  assessment,  and 
implemenlalion  of  learning  environments  and  performance  support 
systems  lor  adults. 

Division  ol  Telecommunications  (DOT)  strives  to  improve  instruction 
Ihrough  Ihe  use  ol  telecommunications  including  television,  radio,  video, 
audio,  recordings,  leleconlerences,  and  interactive  distance  educalion. 

__  ;  MEMBERSHIP  ENROLLMENT  FORM 


DOT  is  concerned  with  Ihe  design,  production,  delivery,  utilization,  and 
evaluation  ol  instructional  lolocommunicalions  materials  and  equipmenl. 


Division  ol  School  Media  Specialists  (DSMS)  enhances  communica- 
tions among  school  media  personnel  and  is  concerned  with  the 
development  evaluation,  and  implemenlalion  ol  school  media  programs 
designed  around  national  guidelines. 

International  Division  (INTL)  onhances  professional  and  personal 
relationships  belwcen  piesenl  and  lulure  international  leaders  and 
encourages  practice  and  research  in  communication  lor  social  and 
economic  developmonl  across  national  lines. 

Industrial  Training  and  Education  Division  (ITED)  Is  Involved  v/ilh 
designing,  planning,  and  managing  training  programs;  and  works  lo 
promote  maximum  ulfealion  of  educalional  techniques  and  media  that 
are  elleclive  in  practical  use. 

Media  Design  and  Production  Division  (MDPD)  provides  an 
international  network  that  locuses  on  enhancing  Ihe  quality  and 
ellecliveness  ol  mediated  communication,  in  all  media  loimals—in 
educational,  governmental,  hospital,  and  corporate  sellings— Ihrough  Ihe 
interaction  of  instructional  Oesigners,  linmeis.  leseaichers,  and 
evalualois  Willi  media  dosigneis  and  produclion  team  specialists  who 
ulili7e  slale  ol  Ihe  art  produclion  skills. 

Research  and  Theory  Division  (RTD)  lacililales  Ihe  design,  execution, 
utilization,  evaluation,  and  dissemination  of  educalional  technology 
research;  promotes  the  applied  and  theoretical  research  on  the  use  of 
educalional  technology;  encourages  the  use  ol  multiple  research 
paradigms  in  examining  issues  relaled  lo  lechnology  in  instruction. 


YES! 


Ennsll  me  U><  ilie  mrinlicisluh«iffinry,dmsi™«,  Ami  ajitionJ  I  have  selected  Iirluw.  I  ir,Htcr«,nul  Ut.it  my  mcmtwrli'ii|i  will  sxicuu 
I  for  12  rtitinllts  finin  the  date  my  payment  is  received. 
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U  International  Regular  Membership,  $87  (surface  mail) 
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G  Comprehensive  Student  Membership,  $60'  I" ee  includes  $14  lor 

TechTrends.  $35  lor  ETRSD,  and  one  Iree  division. 
Q  International  Comprehensive  Student  Membership,  $80* 
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Association  for  Educational  Communications 
and  Technology 


What  Is  AECT? 

AECT  Is  the  only  national,  professional  association  dedicated 
to  tho  Improvement  of  Instruction  through  the  effective  use  of 
media  and  technology.  AECT  assists  Its  members  In  using 
technology  In  their  Jobs  and  to  enhance  the  learning  process. 


Who  belongs  to  AECT? 

U  Media  Specialists 

□  Educators 

□  Librarians 

□  Instructional  designers 

U  Corporate/military  trainers 

□  Learning  resource  specialists 
U  Curriculum  developers 

□  Television  producers  and  directors 

□  Communications  specialist* 

□  Education  administrators 

□  Others  who  requlie  expertise  In  Instructional  technology 


What  ere  AECT  members  Involved  In? 

U  Hypermedia 
U  Interactive  Video 

□  CD-ROM 

□  ca 

□  teleconferencing 

□  Film  &  video  utilization 

□  Telecommunications 

U  Computer  software  8c  hardware 
U  Projection/presentation  products 
U  Intelligent  tutoring  systems 
U  Videodiscs 

□  Distance  learning 
U  And  morel 

AECT's  publications,  convention,  trade  show,  and 
conferences  present  the  leading  edge  on  research  ond 
practical  applications  for  these  and  other  technologies. 


AECT  History 

Unlike  some  olhei  special-Interest  technology  organizations. 
AECT  has  a  long  hfetory.  wtth  over  70  years  In  educational 
technology.  We've  grown  up  wltti  technology,  advocating  Its 
Integration  Into  education  from  films  to  Integration  Into 
education  from  dims  to  overheads  to  Interactive  video  and 
hypermedia. 


AECT  began  as  the  Department  of  Visual  Instruction  at  the 
National  Education  Association  In  1923.  In  the  days  when  visual 
aids  consisted  of  films  and  slides,  h  1947.  as  educators  were 
adapting  technology  used  to  train  World  War  II  service 
personnel  for  the  classroom,  the  name  of  the  organization 
became  the  Department  of  Visual  Instruction  (DAVI).  Twelve 
years  later.  DAVI  become  an  affiliate  of  the  NEA  and  finally  the 
autonomous  association.  AECT,  In  1974. 

Today.  AECT  keeps  an  eye  on  the  future  of  Instructional 
technology  while  assisting  educators  with  the  changes  and 
challenges  that  face  them  now.  AECT  members,  now 
numbering  4500  are  professionals  devoted  to  qualify 
education. 

They  care  about  doing  their  Jobs  better  and  want  to  embrace 
new  methods,  new  equipment,  and  new  techniques  that  assist 
learning. 

Membership  In  AECT  Increases  your  effectiveness,  your 
expertise,  and  your  skills.  These  qualities  In  turn  enhance  your 
professional  Image  and  earning  potential. 


AECT  Affiliates 

AECT  has  47  state  and  16  regional  and  national  affiliates,  and 
has  recently  established  several  chapters  surrounding  major 
universities  and  metropolitan  areas.  These  affiliated 
organizations  add  a  localized  dimension  to  your  AECT 
membership  and  allow  for  more  Interaction  among  your 
colleagues.  For  more  details  on  chapters  and  affllafes  In  your 
area,  contact  the  AECT  National  Office. 


AECT  National  Convention  and  InClTE 
Exposition 

Each  year,  AECT  brings  top  speakers  to  exciting  locations,  and 
presents  over  300  sessions  and  special  events  to  provide  the 
best  training  available  In  the  use  of  media  In  Gducatlon  and 
Instruction.  The  convention  features  the  InClTE  Exposition,  the 
first  trade  show  created  exclusively  for  instructional 
technology  products.  At  InClTE.  you'll  see  computers,  learning 
systems,  software.  Interactive  multimedia,  audiovisual 
products,  films  8i  videotapes,  projectors  and  presentation 
products,  video  equipment,  accessories,  and  more.  The 
Convention  offers  tracks  of  sessions  focusing  on  specific 
Intorest  areas  surrounding  AECT's  nine  Divisions  and  other 
special  Ini  aresfs.  The  convention  ha3  featured  a  Hypermedia 
Strand  and  a  Total  Quality  Management  Track.  In  addition. 
Intensive  full  and  half-day  workshops  are  offered  for  In-deplh 
training  on  the  latest  technology  applications  for  education. 


Research  and  Theory  Division  (RTD)  improves  the 
design,  execution.  utilization,  evaluation,  and  dissemination  of 
educational  technology  research  and  theory;  advises 
educators  on  using  research  results. 
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Abstract 


While  the  ARCS  Model  has  provided  educators  with  a  heuristic  approach  to  generally  increasing 
the  motivational  appeal  of  instruction,  it  may  also  provide  a  model  for  stimulating  and  sustaining  curiosity 
in  particular.  Such  a  model  is  of  great  value  to  educators  who  are  especially  concerned  with  encouraging 
curiosity  as  a  prime  intrinsic  motivator  for  learning  and  to  designers  of  computer-based  systems  who  wish 
to  incorporate  specific  strategies  for  arousing  and  sustaining  the  curiosity  of  users.  This  conceptual  paper 
explores  the  construct  of  curiosity  and  examines  its  relationship  to  all  of  the  ARCS  components. 


Introduction 

"To  be  uncurious  is  to  have  no  antennae  picking  up  the  future's  signals.  To  be  curious  is  to  find 
that  an  inquiry  into  one  subject  soon  has  others  beckoning"  (Weintrub,  1986).  The  importance  of  curiosity 
in  motivating  scholarship  cannot  be  ignored.  Curiosity  has  been  revisited  as  a  topic  of  interest  recently, 
especially  as  it  relates  to  the  design  of  interactive  computer-based  instruction  (eg.,  Rotto,  1994,  Arnone  & 
Grabowski,  1994,  Arnone  &  Grabowski,  1992).  Research  indicates  that  keeping  learners  in  what  Day 
(1982)  calls  a  "Zone  of  Curiosity"  should  result  in  increased  learning  and  a  sense  of  intrinsic  satisfaction. 
Accomplishing  that,  of  course,  is  quite  a  challenge. 

Educators  and  designers  often  incorporate  curiosity-arousing  strategies  into  instruction  in  a 
haphazard  fashion,  when  what  they  really  need  is  a  cohesive  model  with  theoretical  underpinnings  for  the 
selection  and  implemenation  of  such  strategies.  Perhaps  a  new  model  should  be  developed  for  specifically 
addressing  curiosity,  or,  is  it  possible  that  one  already  exists  which  would  work  well  with  the  construct  of 
curiosity  even  though  it  was  developed  for  a  more  general  application  to  motivational  design?  Such  a 
model  would  have  great  value  to  educators  who  are  especially  concerned  with  encouraging  curiosity  as  a 
prime  intrinsic  motivator  for  learning  and  to  designers  of  computer-based  systems  who  wish  to  incorporate 
specific  strategies  for  arousing  and  sustaining  the  curiosity  of  users.  The  purpose  of  this  conceptual  paper 
is  to  first  explore  the  construct  of  curiosity  and  then  to  examine  a  leading  motivational  design  model  for  its 
appropriateness  as  a  heuristic  approach  for  sustaining  curiosity  in  particular. 


Background  on  Curiosity 

The  seminal  mind  in  the  study  of  curiosity  is  Daniel  Berlyne.  He  was  thought  to  have  epitomized 
in  his  own  academic  life  what  is  meant  by  curiosity  and  intrinsic  motivation  (Furedy  &  Furedy,  1981 ). 
Berlyne  died  before  his  work  was  complete  but  his  accomplishments  were  influential  in  much  of  the 
research  in  curiosity  which  followed  including  that  of  Day  (e.g.,  1968,  1982)  and  Maw  and  Maw  (e.g., 
1964,  1965,  1966,  1977).  The  theoretical  groundwork  Berlyne  laid  can  be  seen  in  much  of  today's 
explorations  into  curiosity  both  experimentally  and  conceptually  (e.g.,  Loewenstein,  1994;  Arnone  & 
Grabowski,  1994;  Rotto,  1994;  Arnone  &  Grabowski,  1992).  Other  work  including  that  of  Beswick 
(1968)  and  Langc^-  (1971)  as  well  as  some  of  the  more  recent  researchers  in  the  field  of  intrinsic 
motivation  (e.g.,  Malone,  1981;  Csikszentmilhalyi,  1990;  Keller,  1987)  will  be  explored  in  relation  to  the 
study  of  curiosity. 

In  Berlyne' s  neurophysiological  theory,  cjriosity  is  a  state  of  arousal  brought  about  by  complex 
stimuli  and  uncertainty  in  the  environment,  leading  to  exploratory  behavior.  He  identified  two  forms  of 
exploratory  behavior,  diversive  exploration  and  specific  exploration,  postulating  that  individuals  seeking 
relief  from  changelessness  or  boredom  would  engage  in  diversive  exploration  while  those  wishing  to  resolve 
a  conceptual  conflict  would  engage  in  specific  exploration.  Berlyne  called  the  motivator  for  specific 
exploration  "epistemic  curiosity,"  the  motivational  variable  and  drive  associated  with  an  individual's  "quest 
for  knowledge"  (Berlyne,  1960).  He  postulated  that  the  reception  of  knowledge  and  its  subsequent  rehearsal 
would  reduce  the  drive  (1954). 


Important  to  Berlyne's  theory  is  the  notion  of  collative  variability  which  refers  to  the  complexity 
of  a  stimulus  and  the  level  of  difficulty  it  presents  to  an  individual  in  comparison  to  stimuli  which  are 
already  familiar.  Collative  properties  such  as  unfamiliarity,  novelty,  complexity,  ambiguity,  and 
incongruity  (all  conditions  of  stimulus  uncertainty)  may  increase  arousal  level  and  induce  curiosity. 
Presented  with  a  stimulus  with  a  moderate  degree  of  uncertainty,  an  individual  is  motivated  to  make  sense 
of  this  environment.  He  thus  engages  in  exploratory  behavior  (e.g.  information 
seeking/processing/evaluation)  generally  resolving  any  conceptual  conflict,  and  returns  to  what  Berlyne 
refers  to  as  the  "tonus  level."  It  is  at  this  moderate,  pleasurable  level  of  stimulation  that  an  individual 
functions  most  effectively.  The  optimal  level  of  arousal  (where  the  individual  is  "curious")  is  somewhere 
just  above  the  tonus  level.  Small  increases  in  stimulation  are  considered  pleasurable  because  of  the 
resultant  return  to  one's  tonus  level  one  experiences  once  the  conflict  is  resolved. 

Day  (1982),  a  research  colleague  of  Berlyne,  describes  the  optimal  level  of  arousal  as  one  in  which 
an  individual  enters  a  "Zone  of  Curiosity"  (see  Figure  1).  With  too  much  uncertainty  (caused,  for  example 
by  too  much  stimuli  or  stimu'i  too  high  in  collative  variability),  the  individual  becomes  overwhelmed  or 
anxious  entering  what  he  refers  to  as  a  "Zone  of  Anxiety."  With  too  little  stimulation,  on  the  other  hand, 
the  result  is  disinterest  and  demotivation  often  characterized  by  boredom.  In  either  case,  the  individual  is 
likely  to  withdraw  from  the  exploratory  behaviors  before  resolving  his  conceptual  conflict.  The  degree  of 
importance  an  individual  places  un  the  situation  will  also  play  a  part  in  whether  an  individual's  conflict 
leads  to  curiosity  or  to  fear  and/or  anxiety  (Berlyne,  1960,  Day  &  Berlyne,  1971). 


7f>r»  ofFUlaxllovi 
Unmotivcttd 

hwfifcl«nt 


Efficiency 


Zent  ofCurK>tiiy 

Approach 
E5p{oc«rtkrfi 
ExcltMMnt 
inlwtat 


ZaM-flLfinxifJX 
AvoMaivs* 
Defensive 
DfeintMMtad 
InaRtetont 


Martnass 


ACTIVATION 
(AROUSAL) 


Tonus  L*V»I 

Figure  1:  Day's  Zone  of  Curiosity  (1982)  -  The  Optimal  Level  Of  Arousal 


Both  Day  and  Berlyne  (197 1 )  agree  that  individuals  differ  in  their  tolerance  and  preference  for 
arousal  potential,  suggesting  that  curiosity  is  both  a  state  and  a  trait  variable.  What  one  individual  may 
find  optimally  arousing,  another  may  find  overwhelming  or  the  other  extreme,  understimulating. 
Preference  for  arousal  potential  can  vary  from  individual  to  individual  and  from  occasion  to  occasion.  "How 
much  arousal  potential  of  a  particular  kind  an  individual  will  tolerate  depends  not  only  on  the  level  of 
arousal  tonus  but  also  on  how  promptly  and  easily  he  has  been  able  to  assuage  the  arousal  induced  by 
similar  conditions  in  the  past"  (Berlyne,  1960,  p.  212).  Berlyne  maintains  that  "The  teacher  must, 
therefore,  not  only  be  astute  in  manipulating  the  environment  but  must  also  be  aware  of  individual 
differences  in  tolerance  for  arousal  change"  (p.  328). 

Building  on  Berlyne's  distinction  between  specific  and  diversive  exploration,  Day  developed  and 
validated  tests  of  specific  and  diversive  curiosity.  His  Test  of  Specific  Curiosity  was  a  36-item  measure 
which  he  administered  to  children  in  a  number  of  studies  (e.g.,  Day,  1968).  He  decided,  however,  that  the 
scale  was  too  brief  and  narrow  and  eventually  developed  the  more  comprehensive  and  much  used  Ontario 
Test  of  Intrinsic  Motivation  (OTIM)  (1968). 

Maw  and  Maw  (1964)  operationalized  a  definition  of  curiosity  as  it  relates  to  elementary  school 
children  building  on  the  work  of  Berlyne.  Maw  and  Maw  asserted  that  ". . .  curiosity  is  demonstrated  by  an 
elementary  school  child  when  he: 
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1 .  reacts  positively  to  new,  strange,  incongruous,  or  mysterious  elements 

in  his  environment  by  moving  toward  them,  by  exploring,  or  by  manipulating  them 

2.  exhibits  a  need  or  a  desire  to  know  more  about  himself  and/or  his  environment 

3.  scans  his  surroundings  seeking  new  experiences 

4.  persists  in  examining  and  exploring  stimuli  in  order  to  know  more  about  them"  (p.  31). 

It  should  be  noted  that  their  definition  included  both  diversive  and  specific  curiosity  with  #1,  #2, 
and  #4  relating  to  specific  curiosity  and  #3  relating  to  diversive  curiosity.  Day  also  used  their 
operationalized  definition  in  devising  his  Test  of  Specific  Curiosity  but  since  he  was  primarily 
investigating  specific  curiosity,  he  eliminated  #3.  Maw  and  Maw's  definition  of  curiosity  has  proved  useful 
over  the  years  and  is  frequently  used  even  in  recent  curiosity  studies  (e.g.,  Alberti  &  Witryol,  1994). 

Maw  and  Maw  conducted  many  studies  in  their  development  of  pyschometric  measures  for  trait 
curiosity.  They  tested  the  validity  of  their  measures  by  first  establishing  criterion  groups  of  high  and  low 
curiosity  children  and  then  testing  whether  the  measures  of  curiosity  they  developed  discriminated  between 
the  two  groups.  The  means  used  to  establish  these  groups  included  teacher  judgements,  peer  judgements, 
and  self  judgements  of  curiosity.  The  authors  found  that  teacher  and  peer  judgements  were  not  significantly 
affected  by  factors  such  as  popularity,  race,  or  sex.  Intelligence  had  to  be  statistically  controlled,  however, 
as  both  teacher  and  peer  judgements  were  significantly  related  to  intelligence.  When  the  children  in  the 
tentative  criterion  groups  were  investigated  in  terms  of  their  self-judgements  of  curiosity,  it  was  found  that 
overall  the  children  in  the  higher  curiosity  group  rated  themselves  higher  in  curiosity  than  those  in  the  low 
group.  The  difference  between  the  means  was  significant  at  p  =  .005.  Maw  and  Maw  used  these  groups  to 
validate  items  for  their  various  measures,  one  of  which  was  the  Which  To  Discuss  Test.  This  particular 
measure  also  had  fairly  high  reliability  using  Cronbach's  alpha  as  a  measure  of  internal  consistency,  r  = 
.90.  It  has  been  utilized  in  numerous  studies  (e.g.  Silverstein,  Pearson,  Dunnick  &  Ford,  1981;  Beer, 
1986;  Beer  &  Beer,  1986,  Arnone  &  Grabowski,  1992;  Kerr  &  Beer,  1992;  Arnone  &  Grabowski,  1994). 

Maw  and  Maw  also  used  question-asking  behavior  as  a  measure  of  curiosity.  In  one  study, 
children  were  presented  with  pictures  such  as  an  odd  animal,  for  example,  and  were  then  instructed  to  ask  as 
many  questions  as  they  could  think  of  about  the  pictures.  It  was  found  that  children  who  had  been 
previously  identified  as  high  curious  asked  more  questions  and  had  more  independent  ideas  than  low  curious 
children  (1964).  A  number  of  other  researchers  have  developed  measures  of  curiosity  including  Penny  and 
McCann  (1964),  Naylor's  state  and  trait  measures  (1981),  Vidler  and  Rawan's  Academic  Curiosity  Scale 
(1974,  1975),  Langevin's  Experiential  Curiosity  Measures  (1971),  and  so  on. 

While.  Day,  Maw  and  Maw,  and  many  researchers  have  subscribed  to  the  basic  tenets  put  forth  in 
Berlyne's  theory,  not  everyone  agreed  with  his  theoretical  explanation  of  curiosity.  For  example,  Beswick 
(1968)  proposed  a  cognitive  process  theory  of  curiosity  in  which  an  individual  receives  input  (signals) 
which  is  taken  into  his  cognitive  map  or  category  system  (representing  all  previous  experience  and 
learning),  and  then  is  coded  for  meaning.  If  the  individual  experiences  difficulty  in  the  coding  operation,  he 
would  engage  in  assimilation  (to  modify  the  signal)  and/or  accommodation  (to  modify  the  category  system). 
Accommodation,  then,  represents  new  learning.  Beswick  asserts  that  individual  differences  in  curiosity  are 
a  function  of  category  system  characteristics  and  differences  in  one's  coding  operation  as  they  relate  to  a 
conceptual  conflict.  He  slates  that  different  people  will  have  different  probabilites  of  coding  difficulties 
depending  on  the  particular  situation.  One  uses  different  "strategies"  for  dealing  with  uncertainty. 
Curiosity,  then,  is  a  preferred  cognitive  strategy.  Highly  curious  individuals  have  an  "acquired 
predisposition  to  increase,  prolong,  and  resolve  conceptual  conflicts"  (p.  ).  According  to  Beswick,  a  highly 
curious  person  expects  a  pleasurable  integration  of  the  signal  with  his  or  her  category  system  and  finds  this 
preferred  cognitive  strategy  to  be  intrinsically  rewarding.  Still,  Beswick  uses  many  of  the  same  concepts  as 
Berlyne  and  others  such  as  uncertainty,  conflict,  arousal,  and  "pleasurable"  (in  reference  to  integration  in 
Beswick's  theory  and  stimulation  in  Berlyne's  theory)  to  explain  curiosity. 

H.  Carl  Haywood  also  took  exception  to  Day's  reliance  on  explaining  motivation  only  in  terms  of 
tension-reduction,  suggesting  that  explanations  for  motivation  may  also  be  inherent  in  information 
processing  (1968).  Malone  (1981)  argues  that  it  is  the  kind  of  complexity  or  incongruity  of  the  information 
rather  than  the  amount  that  is  motivating  and  that  there  is  some  optimal  level  of  information  complexity 
that  can  be  tolerated.  Steers  &  Porter  (1987)  state  that  central  to  an  explanation  of  arousal  "is  the  idea  that 
the  individual  cognitively  processes  and  evaluates  a  lot  of  information  and  that  motivation  is  linked 
strongly  to  this  information  processing  activity"  (p.  33).  Olson,  Camp,  and  Fuller  (1984)  found  a  strong 
positive  correlation  between  "need  for  cognition,"  defined  as  the  dispositional  tendency  to  engage  in  and 
enjoy  thinking  (p.  71)  and  curiosity.  Most  recently,  Alberti  and  Witryol  (1994)  have  argued  that  cognitive 
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development  is  enhanced  by  curiosity  and  exploration  because  they  expose  a  person  to  a  "greater  variety  of 
experiences  in  which  learning  can  occur"  (p.  129). 

Given  thai  there  is  a  cognitive  component  to  curiosity,  ihe  importance  of  information  seeking, 
processing,  and  evaluation  behaviors  must  be  considered.  Without  sufficient  and/or  relevant  information,  the 
individual  will  likely  have  great  difficulty  in  resolving  the  cognitive  conflict  the  situation  presents,  thereby 
experiencing  a  dramatic  decrease  in  or  even  extinction  of  curiosity. 

Consider  this  example.  Sarah's  curiosity  is  piqued  when  her  teacher  presents  a  lesson  on  the 
rainforests  of  South  America,  leaving  several  unanswered  questions  at  the  conclusion  of  the  class  (but 
promising  answers  the  following  day).  Immediately  following  the  lesson,  Sarah  goes  to  the  school  library 
media  center  and  peruses  the  online  catalog  for  materials  on  this  topic.  When  she  finds  none,  she  is  highly 
disappointed,  abruptly  ends  her  search,  and  returns  to  her  classroom  unsatisfied.  Only  if  she  persists  (e.g. 
going  to  another  library)  or  there  is  an  external  intervention  (e.g.  the  librarian  assists  her),  might  her 
curiosity  be  rejuvenated  to  its  former  level.  Her  persistence  may  have  been  tempered  by  another  factor, 
importance  of  the  information.  Since  Sarah  had  been  told  that  the  answers  to  these  questions  would  be 
provided  the  next  day,  it  was  likely  not  as  important  as  it  would  have  been  if  the  questions  were  not  soon 
going  to  be  addressed.  Importance  is  addressed  by  the  following  theorist. 

Lanzetta  (1968)  states  that  both  uncertainty  and  importance  contribute  "to  the  magnitude  of 
conflict  and  thus  to  the  strength  of  the  motive  to  search  for  information"  (p.  ),  what  he  calls  the 
"motivation  to  acquire  information"  (p.  ).  As  such,  uncertainty  reduction  is  the  reinforcement  for 
information  searching  and  a  preference  for  sources  of  information  of  high  uncertainty  reduction  potential. 
The  motive  to  reduce  uncertainty  was  also  related  to  curiosity  in  one  of  Kagan's  (1972)  four  basic  human 
motives. 

Malone  (1981)  differentiates  between  sensory  curiosity  and  cognitive  curiosity,  terms  somewhat 
analogous  to  Berlyne's  perceptual  and  epistemic  curiosity.  Sensory  curiosity  involves  enriched  learning 
environments  (e.g.  animations,  brightly-colored  illustrations,  music)  that  provide  sensual  stimulation; 
where,  cognitive  curiosity  is  stimulated  when  a  learner's  environment  is  incomplete,  inconsistent,  and 
unparsimonious,  motivating  him  to  learn  more  in  order  to  improve  his  "cognitive  structure"  (p.  363). 
Malone  recommends  providing  informative  feedback  as  a  means  of  helping  a  cognitively  curious  learner. 

Loewenstein  (1994)  also  argues  the  importance  of  feedback  in  his  recently  developed  "information- 
gap"  theory  of  specific  epistemic  curiosity.  He  acknowledges  the  works  of  earlier  theorists  in  the  field  in 
the  development  of  his  own  theory.  Curiosity,  according  to  Loewenstein,  is  a  feeling  of  deprivation  which 
occurs  as  an  individual  recognizes  a  gap  in  his/her  knowledge  and  is  motivated  to  seek  the  information  that 
will  ameliorate  the  feeling  of  deprivation.  The  information  gap  is  effectively  the  difference  between  "what 
one  knows  and  what  one  wants  to  know"  (p.  87).  He  also  proposed  that  as  a  person's  knowledge  of  a 
particular  domain  increases,  his/her  awareness  of  information  gaps  in  that  knowledge  will  also  increase, 
thus  setting  up  a  necessary  condition  for  curiosity  to  occur.  In  this  perspective,  a  person  would  be  predicted 
to  experience  "a  sudden  increase  in  curiosity  when  the  individual  becomes  focused  on  the  missing 
information  and  then  a  more  gradual  increase  as  he  or  she  approaches  the  goal  of  closing  the  information 
gap"  (p.  89).  Relating  all  this  back  to  feedback,  Loewenstein  asserts  that  one  must  have  an  awareness  of  an 
information  gap  to  experience  curiosity  and  that  accuracy  feedback  is  one  way  of  helping  a  person 
understand  what  he/she  does  not  know.  In  the  absence  of  feedback,  and  with  an  unrealistic  view  of  one's 
knowledge  (i.e.,  an  individual  thinks  he/she  knows  more  about  a  topic  that  he/she  actually  does),  curiosity 
for  the  topic  may  be  decreased  cr  eliminated. 

Whether  one  takes  Berlyne  and  Day's  more  neurophysiological  view,  Beswick  and  others'  more 
cognitive  view,  Malone's  dualistic  view,  or  our  more  integrative  view  that  recognizes  the  importance  of 
both  neurophysiological  and  cognitive  factors  for  the  explanation  of  curiosity,  it  appears  that  the  amount  of 
uncertainty  involved  and  one's  personal  tolerance  for  (or  strategies  for  dealing  with)  a  curiosity-  arousing 
situation  will  affect  an  individual's  response  to  the  source  of  stimulation  (arousal),  input  (signals),  and  so 
on. 

Although  it  is  often  presumed  that  intelligence  and  curiosity  go  together,  most  of  the  studies 
reviewed  showed  low  or  nonsignificant  correlations  between  curiosity  and  IQ.  Langevin  overall  found  IQ  to 
be  distinct  from  curiosity  with  the  exception  of  the  Teacher  Rating  Test  which  had  significant  correlations 
with  the  Otis  Intelligence  Test  and  the  Raven  (r  =  .3524.  r  =  .3263  respectively,  p  =  .01).  Day  and  Berlyne 
(1971)  issue  a  caution  about  teacher  judgements.  Even  if  provided  with  a  list  of  behaviors  that  represent 
curiosity,  there  is  still  a  tendency  to  judge  the  highest  ability  students  as  the  most  curious.  This  was 
shown  in  a  study  which  found  that  teacher  ratings  of  curiosity  related  to  achievement  scores  but  not  to  a 
measure  of  novelty  used  to  assess  curiosity  in  third  and  fifth  grade  children  (Alberti  &  Witryol,  1994). 
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In  a  study  involving  subjects  in  grades  7,  8,  and  9,  Day  (1968)  predicted  that  a  measure  of  specific 
curiosity  which  measured  the  subject's  intent  to  approach  high  levels  of  visual  complexity  and  to  withdraw 
from  low  levels  of  visual  stimulation  would  not  correlate  with  IQ;  he  found  no  significance  (r  =  .01)  as 
expected.  In  another  of  his  experiments  on  specific  curiosity  (1968),  429  subjects  in  grades  7,  8,  and  9 
from  the  same  school  system  participated  in  a  study  in  which  he  administered  his  Test  of  Specific  Curiosity 
and  again  found  no  significance  in  the  correlation  between  his  measure  and  IQ  (r  =  .01). 

No  significant  differences  in  IQ  level  between  high  and  low  curious  subjects  were  found  in  a  study 
by  Maw  and  Maw  (1965)  in  which  differences  in  preferences  for  investigatory  activities  between  high  and 
low  curious  children  were  investigated.  A  study  by  Henderson  and  Gold  (1983)  also  lent  support  for 
curiosity  as  being  distinct  from  intellectual  power.  Henderson  and  Wilson  (1991)  failed  to  support  a 
relationship  between  measures  of  curiosity  and  intelligence  in  preschool  children.  They  offered  one  possible 
explanation  that  perhaps  the  relation  between  these  constructs  was  not  concurrent  but  rather  predictive.  To 
test  this  developmental  view,  they  called  for  longitudinal  studies  that  "investigate  the  ability  of  individual 
differences  in  curiosity  at  time  1  to  predict  changes  in  acquisition  of  knowledge  (e.g.,  intelligence  or 
achievement)  from  time  1  to  time  2"  (p.  173). 

Alberti  and  Witryol's  (1994)  study  of  curiosity  and  cognitive  ability,  based  on  a  similar  view  that 
curiosity  enhances  cognitive  development,  did,  however,  show  an  overall  significant  positive  correlation 
between  a  curiosity  measure  (preference  for  short-term  novelty)  and  standardized  achievement  measures  in 
their  sample  of  elementary  school  aged  children.  If  there  is  such  a  relationship  between  curiosity  and 
cognitive  development,  then  there  is  good  reason  to  encourage  curiosity  at  home  and  in  school. 

One  problem  with  the  study  of  curiosity  is  that  there  doesn't  appear  to  be  any  one  "C"  (crriosity) 
factor  that  we  can  feel  comfortable  about  saying  encompasses  the  construct  entirely.  In  fact,  curiosity 
seems  so  multidimensional  that  it  should  not  even  be  considered  a  unitary  construct.  Yet  it  is  often  treated 
as  such,  especially  in  studies  where  curiosity  is  only  an  ancillary  variable.  Langevin  (1971)  compared  a 
number  of  curiosity  measures  and  two  intelligence  measures  and  found  that  while  most  of  the  measures 
were  distinct  from  intelligence  measures,  several  were  also  somewhat  distinct  from  each  other  indicating 
that  perhaps  their  validity  should  be  addressed.  Henderson  and  Wilson  (1991)  encountered  a  similar  problem 
when  their  measures  of  curiosity  did  not  correlate  significantly  with  each  other.  They  suggested  that  the 
curiosity  measures  may  not  have  been  adequate  indicators  of  curiosity,  or  again  that  their  measures  were 
testing  different  things.  It  is  easy  to  see  from  just  the  brief  sampling  of  theoretical  and  empirical  works 
reviewed  that  curiosity  can  be  considered  both  a  state  and  a  trait,  and  specific  and  diversive.  With  this  in 
mind,  researchers  must  be  cautious  not  to  take  for  granted  a  more  simplistic,  all-encompassing  construct 
called  "curiosity,"  but  rather  to  think  of  it  in  more  complex  terms.  Figure  2  attempts  to  graphically 
represent  the  multifaceted  nature  of  curiosity. 
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Several  writers  and  rc    archers  (e.g.,  Rotto,  1994;  Rezabek,  1994)  have  also  begun  to  explore  the 
Flow  Theory  of  Csikszentmihalyi  (1990)  and  its  relationship  to  motivation  and  curiosity.  Flow  is 
described  as  a  state  of  intense  intrinisic  motivation,  a  natural  high  such  as  what  athletes  describe  when  they 
are  intensely  engaged  in  their  sport.  It  is  an  "optimal  experience"  characterized  by  clear  goals  and  feedback, 
challenges  that  are  matched  to  skills,  a  merging  of  action  and  awareness,  highly  focused  concentration,  a 
sense  of  potential  control,  an  altered  perception  of  time,  a  sense  of  growth,  and  the  engagement  in  an  action 
simply  for  its  own  sake  (Csikszentmihalyi,  1983).  Flow  can  occur  during  work  activities  or  leisure 
activities  but  Csikszentmihalyi  reports  its  occurence  more  often  during  work  activities  than  leisure 
activities  although  individuals  are  more  motivated  during  leisure  activities  (Csikszentmihalyi  &  LeFevre, 
1989).  Flow  experiences  can  happen  at  home,  in  conjunction  with  hobbies,  sports,  and  even  during  more 
passive  activities    anytime  when  challenges  and  skills  are  approximately  equal. 

Could  what  Csikszentmihalyi  calls  optimal  challenge  resemble  what  Berlyne  and  others  have 
referred  to  as  the  optimal  level  of  arousal  necessary  for  curiosity  to  be  sustained?  It  would  seem  that  at  least 
some  aspects  of  sustaining  a  state  of  curiosity  would  resemble  those  in  a  flow  state.  However,  the 
construct  of  curiosity,  at  least  epistemic  curiosity,  involves  some  kind  of  conceptual  conflict  which  must 
be  resolved.  In  flow,  while  challenge  is  necessary,  conflict  is  not.  Malone  (1981)  explains  that 
Csikszentmihalyi  focuses  on  the  role  of  challenge  in  intrinsic  motivation  but  omits  mentioning  the 
importance  of  curiosity  because  he  is  not  dealing  specifically  with  a  learning  context. 

John  Keller's  work  (e.g.  1979;  1983)  has  focused  on  the  importance  of  motivation  in  instruction 
and  learning.  Keller  has  developed  the  ARCS  (Attention,  Relevance,  Confidence,  Satisfaction)  Model  of 
Motivational  Design,  based  largely  on  Porter  and  Lawler's  (1968)  Expectancy-Value  Theory,  but  also 
incorporating  a  number  of  other  motivational  variables  (e.g.  locus  of  control,  need  for  achievement).  Keller 
( 1979)  cites  the  importance  of  curiosity  as  a  motive  to  reduce  uncertainty,  "a  person's  responsiveness  to 
incongruity  and  uncertainty"  (p.  31)  and  uses  it  as  the  primary  theoretical  basis  of  the  attention  component 
of  his  model.  He  recommends  a  judicious  application  of  instructional  strategies  that  arouse,  maintain, 
and/or  increase  curiosity  (e.g.  presenting  novel,  incongruous,  or  paradoxical  events,  posing  a  perplexing 
problem,  and  using  analogies). 


The  ARCS  Model  and  Curiosity:  Is  There  a  Fit? 

While  Keller's  model  is  designed  to  raise  the  overall  motivational  level  for  instruction  (including 
both  intrinsic  and  extrinsic  motivation),  each  component  of  the  model  may  contribute  to  the  arousing  and 
sustaining  of  curiosity.  The  problem  has  been  that  typically  curiosity  is  linked  to  the  attention  component, 
so  that  parents  and  teachers  sometimes  overlook  the  value  of  the  other  components  in  encouraging  curiosity 
as  a  prime  motivator  for  learning.  In  this  section,  we  will  attempt  to  show  the  "fit"  between  the  ARCS 
model  and  both  the  arousal  and  sustaining  of  curiosity. 

Let's  begin  by  briefly  returning  to  the  theoretical  underpinnings  of  the  ARCS  Model.  As  noted 
earlier,  one  of  the  dominant  theories  upon  which  the  ARCS  Model  is  based  is  Expectancy-Value  (E-V) 
theory  (e.g.,  Porter  &  Lawler,  1975).  E-V  theory  posits  that  motivation  occurs  as  a  result  of  both  one's 
expectancy  for  success  and  one's  perception  that  a  particular  activity  will  be  personally  rewarding.  Keller 
makes  a  direct  link  between  the  value  dimension  and  research  in  the  area  of  curiosity  (1983).  Expectancy 
for  success  factors  into  curiosity  as  well.  For  example,  Loewenstein's  Information  Gap  Theory  holds  that 
"people  will  be  more  curious  to  know  something  if  they  think  it  is  knowable"  (p.93).  Keller  states  that  the 
value  dimension  is  most  closely  related  to  his  attention  and  relevance  components  while  the  expectancy 
dimension  of  E-V  theory  is  most  closely  alligned  with  the  confidence  and  satisfaction  components  (Keller, 
1979).  This  distinction  was  supported  in  a  recent  study  by  Small  and  Gluck  (1994). 

Although  curiosity  is  primarily  associated  with  the  attention  component,  we  assert  that  every 
component  of  ARCS  (i.e.,  attention,  relevance,  confidence  and  satisfaction  potential)  contributes  to  an 
explanation  of  the  successful  arousal  and  sustaining  of  curiosity  and  an  eventual  resolution  of  its  underlying 
conflict.  In  order  to  keep  an  individual  in  Day's  (1982)  Zone  of  Curiosity,  it  may  be  necessary  to 
incorporate  all  of  the  components  into  instruction.  Since  learners  bring  many  individual  differences  to  a 
learning  situation,  however,  any  model  is  necessarily  heuristic  as  opposed  to  prescriptive. 

Rotto  (1994)  notes,  if  the  curiosity -arousing  situation  does  not  "show  promise  of  being  useful, 
moderately  non-threatening,  and  of  being  rewarding,  v/e  will  not  be  led  to  explore  it  in  any  extended  way  (p. 
6),"  This  would  seem  to  be  true  if  you  think  of  the  state  of  curiosity  as  being  one  in  which  the  individual 
is  at  an  optimal  level  of  arousal,  thereby  focusing  all  his  attention  toward  a  particular  stimulus.  The 
individual  must  then  experience  a  desire  to  resolve  a  conflict  (i.e.,  the  conceptual  conflict  has  some 
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relevance  to  him),  be  confident  that  the  conceptual  conflict  can  be  resolved  and  that  he/she  is  capable  of 
resolving  it,  and  have  a  sense  that  there  is  potential  for  satisfication  once  the  conflict  is  resolved.  For 
curiosity  to  be  sustained  (e.g.,  the  exploratory  behaviors  persist  for  long  periods  of  time,  and/or  the  learner 
experiences  repeated  occurrences  of  curiosity  within  the  same  teaching/learning  episode),  then,  in  addition  to 
attention,  there  must  be  relevance,  confidence,  and  satisfaction  potential.  Integrating  Day's  (1982)  and 
others  theoretical  view  of  curiosity,  and  Keller's  ARCS  model,  Figure  3  attempts  to  depict  the  relationship 
between  the  ARCS  components  and  arousing  and  sustaining  curiosity. 
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Figure  3:  Adapted  from  Day  (1982) 
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Curiosity  and  Attention.   Keller  (e.g.,  1983)  describes  a  relationship  between  curiosity  and  the 
attention  component  of  his  model  and  suggests  a  number  of  strategies  for  gaining  and  sustaining  attention, 
such  as  novelty,  surprise,  incongruity,  ambiguity,  etc.  These  same  elements  have  been  described  in  prior 
curiosity  research  as  properties  of  collative  variability  which  are  representative  of  uncertainty  and  useful  in 
inducing  curiosity. 

Consider  the  following  example.  At  Miami's  Central  High  School,  Ms.  Adams,  the  chemistry 
teacher,  describes  how,  according  to  scientific  law,  the  combination  of  chemical  X  and  chemical  Y  should 
turn  the  litmus  paper  blue.  Later  in  chemistry  lab,  Jim  conducts  an  experiment  in  which  he  mixes  the  two 
chemicals  and  inserts  litmus  paper,  but  upon  withdrawing  the  paper  Finds  that  it  has  turned  red.  He  is 
surprised  by  this  unexpected  outcome,  his  curiosity  is  aroused  and  he  turns  his  attention  to  searching  for  the 
cause. 

Figure  4  builds  on  Day's  Zone  of  Curiosity  and  represents  a  situation  in  which  there  is  no  learner 
attention.  Learner  attention  is  a  prerequisite  to  the  establishment  of  relevance,  building  of  confidence,  and 
recognition  of  satisfaction  potential.  When  attention  is  absent,  the  learner  (indicated  by  an  X)  never  enters 
the  zone  of  curiosity,  remaining  instead  in  a  zone  of  disinterest. 
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Figure  4  :  No  A  ( curiosity  is  not  aroused) ,  therefore  no  chance  for  R,  C,  or  S, 


Figure  4.  Situation  Lacking  Attention  Prohibits  Relevance,  Confidence,  and  Satisfaction 
Potential 


Curiosity  and  Relevance.  To  keep  one  in  a  zone  of  curiosity  (at  least  when  discussing  state 
epistemic  curiosity),  the  second  ARCS  component,  relevance,  also  comes  into  play.  Perceived  value 
(personal  relevance)  has  been  shown  to  be  significantly  related  to  the  decision  to  seek  additional  knowledge 
in  young  and  middle-aged  adults  (Camp,  Rodrigue,  &  Olson,  1984).  Loewenstein  (1994)  acknowledges  the 
role  preexisting  interests  play  in  elevating  one's  informational  reference  point  which  is  synonomous  with 
what  a  person  wants  to  know.  Curiosity  occurs  when  one's  "informational  reference  point  in  a  particular 
domain  becomes  elevated  above  one's  current  level  of  knowledge"  (p.  89).  This  suggests  that  educators 
increase  their  chances  of  stimulating  curiosity  when  they  give  learners  choices  (reflecting  personal  interests) 
which  is  a  recommended  relevance  strategy  in  the  ARCS  Model.  Loewenstein's  information-gap 
perspective  of  curiosity  also  predicts  that  curiosity  increases  as  knowledge  of  a  domain  increases  given  that 
the  objective  value  of  the  missing  information  remains  the  same  (p.90).  It  would  seem,  then,  that 
relevance  can  be  increased  by  introducing  a  curiosity-arousing  situation  which  has  at  least  some  familiarity 
to  the  learner  (that  is,  the  learner  already  has  some  knowledge  about  it).  The  relevance  component  of  the 
ARCS  Model  reflects  the  importance  of  familiarity  and  tying  instruction  to  the  learner's  experiences. 

Relevance  may  even  occur  naturally  (that  is,  without  any  external  intervention)  as  awareness  of  a 
gap  in  one's  knowledge  becomes  apparent.  Also,  in  line  with  Beswick's  (1968  )  cognitive  processing 
model,  any  stimulus  event  or  "signal"  that  is  at  odds  with  one's  existing  "cognitive  map"  is  likely  to 
automatically  become  more  relevant.  Considering  this,  attention  and  relevance  go  hand  in  hand.  If  an 
attention  strategy  such  as  introducing  incongruity  is  used  for  a  particular  instructional  purpose,  it  should,  in 
addition  to  arousing  curiosity,  lead  to  increasing  the  importance  to  the  learner  of  continuing  her  search  until 
there  is  no  longer  incongruity,  thus  sustaining  curiosity.  This  is  consistent  with  Lanzetta's  (1968) 
emphasis  on  uncertainty  (attention-related)  and  importance  (relevance-related)  as  critical  to  the  motivation  to 
acquire  information. 

Returning  to  our  example,  Jim  continues  to  explore  the  possible  factors  that  contributed  to  the 
result  of  his  experiment  because  he  feels  a  need  to  close  his  information  gap  and  explain  (and  potentially 
correct)  the  unexpected  outcome.  Therefore,  even  when  a  learner's  attention  is  aroused  and  he  feels  confident 
that  he  can  resolve  the  uncertainty  or  incongniity  (and  thereby  satisfy  his  curiosity),  if  relevance  is  lacking 
then  curiosity,  which  is  intrinsic  in  nature,  cannot  be  sustained  (see  Figure  5).  As  a  result,  the  learner 
perceives  no  potential  for  satisfaction  and  becomes  unmotivated  and  disinterested.  Persistence  in  the 
activity  will  likely  be  prolonged  only  if  it  is  connected  to  some  external  motivation,  such  as  a  raise  in  pay 
or  a  high  grade,  a  motive  which  no  longer  can  be  considered  curiosity. 
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Figure  5:  A,  C,  bul  no  R  and  hence  no  S, 


Figure  5.  Attention  and  Confidence  Without  Relevance  Results  in  Lack  of  Satisfaction  Potential. 


Curiosity  and  Confidence.  Even  if  a  learner  finds  particular  instructional  materials  or  events 
initially  arousing  and  somewhat  relevant,  it  is  unlikely  that  curiosity  will  be  sustained  without  some 
confidence  that  she  will  be  able  to  resolve  the  conflict.  If,  on  the  other  hand,  the  learner  feels  confidence 
that  she  can  resolve  a  particular  conceptual  conflict,  she  is  likely  to  stay  in  a  zone  of  curiosity, 
manipulating  or  exploring  the  environment.  For  example,  dealing  with  ambiguity  entails  seeking  needed 
information  through  exploratory  and  epistemic  behavior,  but  as  Berlyne  (1960)  notes  "this  means  that  the 
conflict  must  be  faced  and  born  for  a  while"  (p.216).  A  study  Berlyne  described  in  his  famous  book 
"Conflict,  Arousal,  and  Curiosity"  found  a  positive  correlation  between  tolerance  for  ambiguity  and  self- 
confidence.  Tolerance  for  complexity  could  also  play  a  pdrt  in  whether  curiosity  is  sustained  or 
extinguished.  Yerkes  &  Dodson  (1908)  found  that  if  a  task  is  too  difficult,  motivation  decreases.  If  a 
conceptual  conflict  proves  too  complex  or  difficult,  one  could  likewise  expect  curiosity  to  decrease.  The 
confidence  component  of  the  ARCS  Model  suggests  strategies  for  dealing  with  difficulty  level  of  materials 
which  could  be  modified  to  address  curiosity  in  particular. 

Anxiety  may  play  a  part  in  undermining  one's  confidence  and  in  cutting  short  one's  time  in  the 
zone  of  curiosity  to  which  Day  (1982)  refers.  Anxiety  reduction  is  also  addressed  under  this  component  of 
Keller's  Model  with  one  suggestion  being  that  confirmational  and  corrective  feedback  will  assist  in  reducing 
anxiety;  this  would  result  in  extending  the  time  one  is  willing  to  spend  in  the  zone  of  curiosity.  The  use  of 
informative  feedback  has  been  associated  with  the  reinforcement  of  curiosity  (Malone,  198 1).  If,  in  our 
example,  Ms.  Adams  provides  Jim  with  some  verbal  cues  about  his  perplexing  problem  and  suggests 
additional  books  and  other  information  resources,  Jim  may  feel  more  confident  that  he  will  resolve  his 
curious  situation. 

While  flow  theory  addresses  intrinsic  motivation  and  not  specifically  curiosity,  it  could  be 
hypothesized  that,  in  some  cases,  a  sustained  state  of  curiosity  may  evolve  into  a  "flow"  state  in  the  process 
of  resolving  a  conceptual  conflict.  As  our  friend  Jim  delves  into  the  various  information  resources,  he  may 
become  so  absorbed  that  he  experiences  flow.  Challenges  and  skills  are  important  to  flow  theory,  challenge 
and  curiosity  have  been  linked  (Malone,  1981),  and  they  both  seem  to  be  related  to  the  concept  of  self- 
confidence.  In  the  confidence  component  of  the  ARCS  Model,  challenge  levels  (e.g.,  deciding  on  an 
appropriate  challenge  level  for  a  particular  audience)  are  addressed,  as  well  as  building  feelings  of 
competence  and  positive  expectations  for  success. 

So,  if  a  situation  occurs  in  which  both  attention  and  relevance  are  present  but  the  learner  believes 
the  curiosity-arousing  situation  is  too  difficult  or  complex,  the  learner  may  lack  confidence  to  continue. 
Without  confidence,  and  in  the  presence  of  a  stimulus  with  a  high  degree  of  uncertainty,  the  individual  will 
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likely  fall  back  into  a  zone  of  disinterest  or  fall  forward  in  a  zone  of  anxiety,  eventually  withdrawing  from 
the  exploratory  behaviors  as  the  curiosity  quickly  diminishes.  In  this  scenario,  satisfaction  potential 
disappears  (see  Figure  6). 


Figure  6.  Attention  and  Relevance  Present  But  No  Confidence  And  Therefore  No 
Satisfaction  Potential. 


Curiosity  and  Satisfaction  Potential.  Satisfaction  potential  must  be  high  in  order  for  an  individual 
to  remain  in  a  curiosity  state.  If  there  is  only  a  slim  chance  for  satisfying  one's  curiosity,  it  is  unlikely 
that  an  individual  will  expose  himself  to  a  curiosity  inducing  situation  (Loewenstein,  1994).  Since  the 
curiosity-invoking  situation  produces  an  aversive  state  of  tension  and  uncertainty,  the  potential  for 
satisfaction  resulting  from  its  reduction  is  correspondingly  high.  According  to  Berlyne  and  Day,  upon  the 
actual  resolution  of  a  curiosity  arousing  situation  the  individual  returns  to  his/her  tonus  level.  Loewensteii, 
writes:  "The  pleasure  derived  from  satisfying  curiosity  provides  a  simple  explanation  for  voluntary  curiosity 
seeking.  It  is  perfectly  sensible  for  people  to  expose  themselves  to  curiosity-inducing  situations  if  the 
expected  incremental  pleasure  from  obtaining  the  information  compensates  for  the  aversiveness  of  the 
curiosity  itself  (p.90).  (This  statement  can  also  be  related  to  the  confidence  component  of  the  ARCS 
Model  that  addresses  positive  expectations  for  success  as  a  motivator.)  The  fit  with  Keller's  Model  comes 
as  he  addresses  intrinsic  reinforcement  under  the  satisfaction  component.  For  Jim,  then,  an  understanding  of 
the  cause  of  his  experimental  outcome  has  the  potential  for  satisfaction  (in  keeping  with  several  theoretical 
approaches)  in  the  form  of  tension  reduction,  cognitive  integration  or  closing  his  information  gap,  intrinsic 
feelings  of  accomplishment  and  competence,  and  praise  from  his  teacher  as  an  extrinsic  bonus  which  is  of 
less  interest  to  us. 

Only  after  the  individual  resolves  the  curiosity-invoking  conflict  will  actual  satisfaction  occur.  In 
our  example,  Jim  experiences  satisfaction  when  he  reads  in  one  of  his  sources  that  if  even  a  small  amount 
of  chemical  Z  is  added  to  chemicals  X  and  Y,  a  different  chemical  reaction  will  occur.  Jim  remembers  that 
during  the  last  lab  session,  he  had  neglected  to  clean  his  beaker  which  had  contained  chemical  Z.  As  a 
result,  the  beaker  contained  residue  of  chemical  Z  which,  when  mixed  with  the  two  chemicals  in  the 
experiment,  caused  the  litmus  paper  to  turn  red  rather  than  blue.  After  thoroughly  cleaning  the  beaker,  Jim 
tries  the  experiment  again  and  obtains  the  expected  result. 
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Figure  6:  A,  R,  but  no  C,  S( 
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Figure  7  depicts  the  presence  of  all  ARCS  components.  The  result  would  be  a  sustaining  of 
curiosity  followed  by  a  return  to  the  pleasurable  tonus  •  -  el  (equated  to  the  satisfaction  component  of 
ARCS)  once  uncertainty  is  reduced  (e.g.,  by  resolving  a  conceptual  conflict). 
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Figure  7:  A,  R,  C,  S,„,are  all  present 


Figure  7.  All  ARCS  Components  Present,  Satisfaction  Occurs. 


Design  Recommendations 

In  this  section  of  our  paper,  we  include  recommendations  for  arousing  and  sustaining  curiosity  in 
learning  contexts  using  the  ARCS  Model  of  Motivational  Design.  For  example,  strategies  in  keeping 
with  the  first  two  components  of  ARCS,  attention  and  relevance,  can  be  used  to  introduce  a  conceptual 
conflict  that  has  at  least  some  relevance  (any  stimulus  which  differs  from  schema  already  accepted  by  the 
individual  may  suddenly  become  more  relevant  as  he/she  struggles  to  integrate  it  into  existing  schema). 
The  third  component  comes  into  play  as  we  design  strategies  which  give  the  learner  the  confidence  that  he 
or  she  will  be  capable  of  resolving  whatever  conceptual  conflict  the  instance  of  curiosity  has  engendered. 
Satisfaction  potential  in  the  context  of  curiosity  refers  to  an  individual's  perceptions  about  the  potential 
intrinsic  rewards  that  will  result  from  resolving  the  curiosity-invoking  situation.  Satisfaction,  the  last 
ARCS  component,  occurs  when,  after  sufficient  exploration  (while  in  the  zone  of  curiosity),  one  resolves 
the  conceptual  conflict  and  returns  to  the  pleasurable  tonus  level.  Satisfaction  can  take  many  forms 
including  external  rewards,  but  when  stimulating  curiosity  is  the  motivational  goal,  it  is  in  intrinsic 
satisfaction  as  a  natural  by-product  of  a  curiosity-arousing  situation  that  we  are  particulaiiy  interested. 

The  following  section  includes  a  number  of  curiosity-arousing  and  maintaining  strategies  related  to  each  of 
the  ARCS  components  that  may  be  incorporated  into  an  instructional  design. 
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ARCS  COMPONENTS 

RECOMMENDATIONS  FOR  AROUSING  AND 
SUSTAINING  CURIOSITY 

ATTENTION 

•Use  collative  properties  such  as  complexity,  novelty,  incongruity, 
contradiction,  and  ambiguity  to  provoke  a  cognitive  conflict. 

•Create  an  atmosphere  where  learners  feel  comfortable  about  raising 
questions  and  where  they  can  test  their  own  hypotheses  through 
discussion  or  brainstorming. 

•Be  aware  of  the  degree  of  stimulation  which  is  entered  into  the 
learning  situation.  Anxiety  may  be  induced  if  the  stimulus  is  too 
complex,  too  uncertain,  too  novel.  Boredom  may  result  if  there  is 
too  little  stimulation. 

•Be  senstive  to  individual  differences  in  tolerance  for  curiosity  arousal 
(notice  which  individuals  approach  new  and/or  different  stimuli  and 
which  display  avoidance  behaviors).  This  should  help  you  in 
determining  how  much  uncertainty  to  introduce  when  attempting  to 
arouse  curiosity. 

•Help  learners  become  aware  of  their  information  gaps  in  a  particular 
topic  area. 

•Minimize  distractions  that  may  have  the  effect  of  eliminating 
curiosity  once  aroused. 

RELEVANCE 

•Model  curious  behaviors  to  increase  learners'  perceptions  of  the 
importance  of  curiosity. 

•Allow  students  to  become  engaged  and  curious  about  topics  or 
interests  in  which  they  already  have  some  knowledge. 

•Have  learners  think  about  how  they  might  feel  once  they  have 
resolved  the  conceptual  conflict. 

•Make  statements  that  help  learners  to  see  the  potential  value  in 
persisting  in  a  curiosity-arousing  situation  until  the  necessary 
information  has  been  round.  Have  tnem  set  tneir  own  personal 
information  goals. 

•Provide  opportunities  for  learners  to  choose  alternative  strategies  for 
exploring  curiosity-stimulating  situations. 
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CONFIDENCE 

•Make  the  challenge  or  conflict  in  line  with  learners'  skills  and 
abilities  and  at  the  appropriate  level  of  complexity. 

•Provide  informative  feedback  that  will  help  learners  see  a  realistic 
picture  of  their  information  gaps  and  reinforce  their  curiosity 
(reducing  possible  anxiety). 

•Help  learners  have  positive  expectations  for  success  based  on  having 
clear  goals  (e.g.,  learner  has  a  plan  for  seeking  the  necessary 
information  to,  as  Loewenstein  (1994)  would  say,  close  his/her 
information  gap). 

•Encourage  exploration  and  give  learners  opportunities  to  explore 
curiosity  arousing  situations  in  their  own  ways. 

•Support  information-seeking  activities. 

•Allow  time  for  learners  to  persist  in  their  explorations  or 
information  seeking. 

SATISFACTION 

While  satisfaction  is  a  natural  by-product  of  a  curiosity-arousing 
situation  once  the  conceptual  conflict  has  been  resolved,  there  are  a 
number  of  strategies  taken  from  Keller's  Model  which  could  provide 
additional  satisfaction  while  setting  the  stage  for  future  curiosity 
behaviors  to  occur. .  . 

•Bring  attention  to  the  intrinsic  enjoyment  that  learners  feel  as  a 
result  of  reducing  the  uncertainty  connected  with  curiosity. 

•Be  enthusiastic  about  the  learners'  accomplishments  as  they  relate  to 
curiosity. 

•Give  learners  an  opportunity  to  reflect  on  their  curiosity  arousing 
episodes  and  their  resolution. 

•Acknowledge  any  special  challenges  that  may  have  been  encountered 
in  the  process  of  resolving  information  uncertainty. 

•Provide  the  learner  with  additional  materials  about  related  areas  of 
interest. 

•Inform  learners  if  possible  about  ways  in  which  they  might  continue 
to  explore  the  area  which  was  the  source  of  their  recent  curiosity. 

•If  one  learner  has  resolved  his/her  uncertainty  before  others,  allow 
him/her  to  provide  guidance  and  informative  feedback  to  others  who 
are  not  yet  there. 

(Note  the  concentration  on  instrinsic  reinforcement  as  opposed  to 
extrinsic  rewards) 
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Summary 


While  the  ARCS  Model  has  provided  educators  with  a  heuristic  approach  to  generally  increasing 
the  motivational  appeal  of  instruction,  it  may  also  provide  a  model  for  stimulating  and  sustaining  curiosity 
in  particular,  certainly  an  important  challenge  to  today's  educators  and  instructional  designers.  This  paper 
has  reviewed  some  of  the  research  on  curiosity,  examined  the  relationship  of  curiosity  to  each  of  the  ARCS 
components,  and  provided  some  ideas  for  stimulating  and  sustaining  curiosity  in  learning  contexts. 

One  of  the  greatest  challenges  of  educators  today  is  fostering  intrinsic  motivation  in  learners  - 
encouraging  a  lo"e  of  and  desire  for  lifelong  learning.  Research  that  provides  a  framework  for  designing 
instruction  that  stimulates  and  sustains  curiosity  may  provide  one  giant  step  in  that  direction. 
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Study  is  a  key  concept  in  making  design  more  fruitful  in  education.  We  propose  that  what  students 
are  acting  when  they  construct  knowledge  is  studying.  For  several  years,  Teachers  College,  Columbia 
University  and  the  Dalton  School  (an  independent  school  in  New  York  City),  have  been  collaborating  on 
the  Dalton  Technology  Project.  The  goal  of  the  project  is  to  use  networked  multimedia  workstations  to 
provide  Study  Support  Environments  (SSEs).  Creating  SSEs  allows  us  to  create  "a  place  for  study  in  a 
world  of  instruction"  (McClintock,  1971).  The  core'of  study  is  the  hermeneutic  activity  of  constructing 
interpretations.  From  this  perspective,  the  basis  for  cognition  is  interpretation  based  on  background 
knowledge  and  beliefs  (Heidegger,  1962;  Winograd  and  Flores,  1986).  Thus,  the  key  consideration  in 
designing  a  SSE  is  fostering  the  construction  of  interpretations  based  on  observations  and  background 
contextual  information.  In  this  paper,  we  describe  a  framework  for  SSE  design  and  describe  its  application 
to  a  specific  SSE  created  as  part  of  the  Dalton  Technology  Plan.  After  describing  the  SSE  we  report  an 
evaluation  that  demonstrates  its  effectiveness. 

Interpretation  Construction  (ICON)  Design  Model 

Much  of  the  effort  in  the  Dalton  Technology  Project  has  gone  into  developing  particular  study 
systems  for  different  subject  areas,  but  we  have  also  been  working  to  specify  what  the  appropriate  design 
principles  would  be  for  this  approach.  As  reported  earlier  (Black  and  McClintock,  1995;  Black,  McClintock 
and  Hill ,  1994)  this  constructivist  design  approach  seems  to  be  captured  by  the  following  seven  principles 
comprising  the  Interpretation  Construction  (ICON)  design  model: 

1.  Observation:  Students  make  observations  of  authentic  artifacts  anchored  in 

authentic  situations 

2.  Interpretation  Construction:  Students  construct  interpretations  of  observations  and 

construct  arguments  for  the  validity  of  their  nterpretations 

3.  Contextualization:  Students  access  background  and  contextual  materials  of 

various  sorts  to  aid  interpretation  and  argumentation 

4.  Cognitive  Apprenticeship:  Students  serve  as  apprentices  to  teachers  to  master 

observation,  interpretation  and  contextualization 

5.  Collaboration:  Students  collaborate  in  observation,  interpretation  and 

contextualization 

6.  Multiple  Interpretations:  Students  gain  cognitive  flexibility  by  being  exposed 

to  multiple  interpretations 

7.  Multiple  Manifestations:   Students  gain  transferability  by  seeing  multiple 

manifestations  of  the  same  interpretations 

Some  of  these  constructive  design  principles  are  adaptations  from  proposals  by  others.  For 
example,  the  Cognitive  Apprenticeship  principle  comes  from  Collins,  Brown  and  Newman  (1988),  the 
Multiple  Interpretations  one  from  Spiro,  Feltovich,  Jacobson  and  Coulson  (1992),  and  the  Collaboration 
one  from  Johnson,  Johnson,  Holubec  and  Roy  (1984).  The  Observation  principle  is  a  combination  of 
recommendations  by  Brown,  Collins  and  Duiguid  (1989)  and  the  Cognition  and  Technology  Group  at 
Vanderbilt  (1990),  but  our  focus  on  authentic  artifacts  is  unique.  Further,  our  emphasis  on  Interpretation 
Construction,  Contextualization,  and  Multiple  Manifestations  is  distinctive. 


An  Example  SSE 

To  illustrate  the  application  of  this  design  framework,  we  describe  an  SSE  program  created  for  the 
Dalton  Technology  Plan.  Specifically,  we  describe  how  these  constructive  design  principles  apply  to  the 
Galileo  program  used  in  1 1th  and  12th  grade  science  (particularly  for  students  not  scientifically  oriented)  at 
the  Dalton  School. 
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In  the  Galileo  program,  students  study  astronomy  and  science  in  general  by  using  observations  of 
telescopic  plates  and  a  computer  simulation  of  the  sky  to  construct  and  test  interpretations  of  astronomical 
phenomena.  Students  examine  and  make  measurements  on  photographic  plates  from  observatory 
telescopes  and  computer  simulations  of  the  sky  (Observation),  then  relate  these  analyses  to  reference 
materials  (Contextualization)  containing  what  is  know  about  astronomical  objects  (i.e.,  stars,  planets, 
etc.).  The  teacher  initially  talks  through  how  he  would  analyze  and  interpret  examples  of  such  astronomical 
data  (Cognitive  Apprenticeship)  then  the  students  form  groups  to  work  on  some  data 
(Collaboration),  while  the  teacher  coaches  and  advises  them  as  they  proceed.  The  students  develop  their 
own  hypotheses  and  test  them  against  the  astronomical  data  (Interpretation  Construction).  Students 
defend  their  hypotheses  using  their  analyses  and  reference  materials  both  within  and  between  the  groups,  and 
such  argumentation  together  with  background  readings  exposes  them  to  various  ways  to  interpret  the  data 
(Multiple  Interpretations).  As  they  proceed  through  the  course,  the  students  see  how  basic  principles 
of  astronomy,  physics  and  chemistry  can  be  used  to  make  sense  of  different  sets  of  astronomical  data 
(Multiple  Manifestations). 


An  Evaluation  of  GenerU;  Skill  Learning  with  the  SSE 

Since  we  believe  that  interpretation  is  central  to  cognition  and  learning,  we  evaluated  whether  the 
Galileo  program  would  increase  students'  interpretation  skills.  Specifically,  we  tested  whether  the  students 
who  had  been  through  this  program  could  make  observations  and  interpretations  in  a  completely  new  area 
better  than  students  who  had  not  been  through  the  programs.  For  these  studies,  we  chose  an  area  unlikely 
to  be  familiar  to  precollege  students  --  namely,  cognitive  psychology. 

In  this  evaluation  study,  the  1 1th  and  12th  grade  students  who  had  been  through  the  Galileo 
program  were  compared  to  a  comparable  group  on  how  well  they  could  interpret  and  link  three  related 
cognitive  psychology  studies  and  their  underlying  principles.  The  students  were  given  booklets  containing 
descriptions  of  basic  observations  made  in  these  three  psychology  studies  together  with  various 
informational  resources  including  relevant  and  irrelevant  background  material.  The  students  were  given 
three  hours  to  perform  the  task  and  write  a  final  report.  These  reports  served  as  a  measure  of  the  students 
abilities  to  recognize  particular  patterns  in  the  data,  argue  or  explain  the  causes  and  effects  of  these  patterns, 
as  well  as  represent  the  data  to  support  or  refute  their  interpretations. 

Method 

Participants 

The  experimental  group  consisted  of  46  1 1th  and  12th  grade  students  at  the  Dalton  School  who  were 
at  the  end  of  the  Galileo  course.  The  control  group  was  33  Dalton  10th  and  1 1th  grade  students  who  planned 
to  take  the  Galileo  course  during  the  following  year. 


Materials 

The  participants  were  provided  with  copies  of  data  from  three  cognitive  psychology  studies.  The 
participants  were  told  that  in  all  three  of  the  studies  they  were  reading  about,  the  researchers  select  six 
students  and  have  them  memorize  a  list  of  12  Subject- Verb-Object  sentences  (propositions).  However,  the 
studies  differ  in  the  following  ways: 

Study  One.  One  full  day  after  remembering  the  list,  participant  memory  is  tested  with  items  on 
computer  screen,  the  propositions  appear  in  the  same  form  as  they  were  memorized.  The 
computer  records  how  long  it  take  each  participant  to  respond  with  an  affirmative  answer. 

Study  Two.  A  different  set  of  six  students  are  asked  to  recall  the  propositions  they  memorized  a 
day  before;  however,  this  time,  they  are  tested  with  the  passive  voice  version  of  the  sentences 
instead  of  the  active  voice  version  they  memorized.  Once  again,  the  computer  records  how  long  it 
takes  each  participant  to  respond  with  an  affirmative  answer. 

Study  Three.  Yet  another  set  of  six  students  are  asked  to  recall  the  propositions  they  memorized 
earlier;  however,  in  this  study,  participants  are  tested  with  paraphrases  of  the  sentences 
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memorized  and  asked  to  respond  "yes"  if  the  test  sentence  as  essentially  the  same  meaning  as  the 
memorized  sentence. 

Upon  reading  these  three  studies  and  the  background  materials  provided,  Daiton  students  were  asked 
to  interpret  the  data  using  background  materials  provided  as  well  as  other  outside  sources.  The  background 
readings  included  both  relevant  information  about  information  processing  systems  as  well  as  propositional 
network  theory.  Irrelevant  information  from  philosophy  was  also  included  as  a  distractor. 

Procedure 

Administering  the  Materials  and  Collecting  Student  Reports. 

The  study  was  conducted  in  one  3-hour  session.  First,  the  experimenter  passed  out  the  assignment 
booklets  and  the  teacher  read  the  instructions  on  the  first  page  of  the  assignment  booklet.  After  the 
instructions,  the  students  proceeded  to  work  on  the  assignment  in  their  groups.  While  doing  the 
assignment  the  students  were  free  to  use  any  of  the  resources  in  the  Daiton  School  building  (computers, 
libraries,  etc.)  including  asking  the  experimenter  clarification  and  information  questions  (the  same 
experimenter  and  teacher  conducted  all  sessions).  At  the  end  of  the  3-hour  period  the  students  handed  in 
their  reports  and  all  the  work  they  had  done  in  folders. 

Analysis  of  Student  Reports 

The  file  folders  from  both  the  Galileo  and  prt-Galileo  groups  were  evaluated  along  the  following 
there  dimensions:  pattern  recognition,  explanation  and  argumentation,  and  data  representation.  These 
dimensions  were  weighted  2:3:1  based  upon  levels  of  difficulty.  More  specifically,  groups  could  earn  up  to 
20  points  for  pattern  recognition,  30  points  for  explanation  and  argumentation,  and  10  points  for  data 
representation.  Extra  credit  points  were  awarded  for  plausible  recommendations  for  follow-up  studies  on  the 
cognitive  psychology  principles  tested;  however,  a  majority  of  the  students  did  not  address  this  aspect  of  the 
data. 

Using  our  coding  system,  we  were  able  to  weigh  answers  in  terms  of  difficulty  as  well  as 
plausibility,  the  "optimal  responses  for  pattern  recognition,  explanation  and  argumentation,  and  data 
representation  are  as  follows: 

Pattern  Recognition 

Study  One.  Students  should  recognize  that  the  response  time  increases  with 
the  number  of  propositions  per  subject.  For  example,  if  lawyer  has  e  propositions  and 
doctor  has  2  propositions  then  subjects  will  take  longer  to  remember  a  proposition  about 
the  lawyer  than  the  doctor.  They  are  also  expected  to  report  the  means.  Partial  credit  is 
given  for  alternative  patterns.  Maximum  Points:  4. 

Study  Two.  Students  should  note  that  the  positive  relationship  between  number 
of  propositions  per  subject  and  response  time  still  holds.  They  should  also  recognize  that 
study  two  takes  longer  than  study  one.  Again,  means  should  be  reported  and  partial  credit 
is  given  for  alternative  patterns.  Maximum  Points:  7. 

Stud\L Three.  Students  should  recognize  that  the  positive  relationship  between 
number  of  propositions  per  subject  and  response  time  still  holds.  They  should  also  note 
that  study  three  takes  longer  than  study  one  and  study  two.  Ideally,  students  should  note 
that  study  three  has  a  steeper  climb  (i.e.,  slope)  than  the  other  two  studies.  Means  should 
be  reported  and  partial  credit  is  given  for  alternative  patterns.  Maximum  Points:  9. 

Explanation  and  Argumentation 

Study  One.  In  this  study,  the  optimal  respones  would  relate  the  information 
processing  and/or  propositional  network  theories  to  the  pattern  recognized.  Partial  credit 
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was  given  for  responses  favoring  individual  differences.  Testing  hypotheses  was  also 
credited.  Maximum  Points:  6. 


Study  Two.  Students  were  supposed  to  relate  the  increase  in  mean  response  time 
to  the  stage  of  transforming  a  proposition  from  passive  to  active  voice.  They  should  have 
also  noted  that  it  takes  time  to  match  the  transformed  proposition  to  the  database. 
Again,  students  were  given  points  for  appropriately  using  information  processing  theory 
or  propositional  network  theory  to  support  their  hypotheses.  Partial  credit  was  given  for 
responses  favoring  individual  differences.  Regardless  of  final  outcome,  hypothesis  testing 
was  also  given  partial  credit.  Maximum  Points:  12. 

Study  Three.  Students  were  given  credit  for  discussing  how  the  overall  increase 
in  mean  response  time  is  due  to  an  increase  in  search  time,  which  is  caused  when  a 
synonym  for  the  original  proposition  is  put  in  the  recall  test.  They  also  should  have 
noted  that  the  increased  time  is  due  to  the  time  required  to  match  the  synonym  against  the 
database.  Students  were  given  credit  for  using  relevant  background  readings.  Partial  credit 
was  given  for  responses  favoring  individual  differences.  Students  were  also  given  partial 
credit  for  reporting  their  hypothesis  test  and  results.  Maximum  Points:  12. 

Data  Representation 

Groups  were  given  credit  for  creating  spreadsheets  that  reported  means  and/or 
ratios,  bar  graphs  that  appropriately  represented  the  data,  propositional  network  diagrams, 
and  alternative  ways  of  representing  the  data  to  support  explanations  of  trends.  Ideally 
students  would  have  been  creating  graphs  that  reflected  the  differences  between  studies. 
By  plotting  these  trends,  students  would  have  recognized  that  the  intercepts  of  study  one 
and  study  three  are  the  same  but  their  slopes  are  different.  Additionally,  the  intercept  of 
study  one  and  study  two  are  different  but  their  slopes  are  the  same.  Representations  of 
these  trends  were  likely  to  have  improved  both  pattern  recognition  and  argumentation 
scores.  Maximum  Points:  10. 

Results 

Table  1  presents  the  results  of  the  data  analyses.   As  the  first  column  in  Table  1  shows,  in  total  the 
Galileo  group  scored  33%  higher  than  the  pre-Galileo  group  (26. 1  vs.  19.6  -  out  of  60  possible),  and  this 
is  statistically  a  very  significant  difference,  f(77)=4.56,  p<.001. 

Table  1 

Quantitative  Analysis  of  Reports  Written  by  Students 
in  the  Galileo  Group  and  the  Control  Group 

Total  Pattern  Explanation   &  Data 

Recognition      Argumentation  Representation 

Galileo  Group  26.1  9.8  14.2  2.1 

Control  Group  19.6  7.7  11.8  9.1 

This  significantly  superior  performance  of  the  Galileo  group  also  occurred  in  all  three  of  the 
component  scores:  namely,  in  Pattern  Recognition  (9.8  vs.  7.7  --  out  of  a  possible  20),  r(77)=  2.40,p<.01 ; 
in  Explanation  and  Argumentation  (14.2  vs  1 1.8  --  out  of  a  possible  30),  r(77)=1.69,  p<.05;  and 
particularly  in  Data  Representation  where  the  pre-Galileo  control  group  effectively  got  0  (specifically,  0. 1 ) 
whereas  the  Galileo  group  got  21%  of  the  possible  (2.1  out  of  10  possible),  f(77)=6.14,p<.001. 
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Conclusions 


We  have  proposed  an  approach  to  constructivist  design  (ICON)  that  makes  interpretation 
construction  of  authentic  artifacts  in  the  context  of  rich  background  materials  the  central  focus.  We  have 
shown  how  this  approach  can  be  applied  to  design  a  Study  Support  Environment  for  teaching  science  and 
scientific  reasoning  to  students  not  scientifically  oriented  We  have  also  shown  that  in  addition  to  learning 
specific  content,  students  using  these  programs  acquire  generalizable  interpretation  and  argumentation  skills. 
Thus,  our  constructivist  design  framework  is  useful  both  for  guiding  design  and  for  producing  valuable 
learning  results. 
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Task  analyses  have  traditionally  been  framed  in  terms  of  overt  behaviors  performed  in  accomplishing  tasks 
and  goals.  Recent  research  has  described  the  central  role  of  cognitive  factors  involved  in  performing  jobs 
ranging  from  dairy  truck  loading  (Scribner,  1984)  to  paper  pulp  manufacturing  (Zuboff,  1988),  ship 
navigation  (Hutchins,  1990)  and  software  debugging  (Riedl,  et  al.,  1991 ;  Flor  and  Hutchins,  1991 ). 
Pioneering  work  at  the  Learning  Research  and  Development  Center  looked  at  what  contribution  a  cognitive 
analysis  might  make  to  current  task  analysis  procedures  (Glaser  et  al,  1986),  since  traditional  task  analysis 
methods  neither  elicit  nor  capture  cognitive  components  of  the  job.  In  the  current  paper,  we  describe  our 
approach  to  cognitive  task  analysis,  and  report  some  initial  case  studies  trying  out  our  methods. 

Specifically,  our  approach  has  sought  to  capture  two  aspects  of  a  job  that  have  been  generally  overlooked 
by  traditional  task  analysis:  (a)  describe  job  performance  not  only  in  terms  of  overt  behaviors,  but  also  in 
terms  of  the  underlying  knowledge  content  and  thinking  processes,  (b)  capture  not  only  the  formal  but  also 
the  informal  aspects  of  the  job. 

Drawing  upon  basic  research  in  Cognitive  Science  we  analyze  knowledge  about  jobs  into  four  knowledge 
representation  types,  namely  factual  knowledge,  imagistic  knowledge,  procedural  knowledge  and  mechanism 
(or  mental  model-type)  knowledge  (Black,  1992). 

Factual  knowledge  gets  at  "knowing  that  something  is  the  case".  It  is  best  represented  as  a 
propositional  network  (Anderson  1983).  The  basic  unit  for  this  kind  of  knowledge  is  the  proposition, 
which  is  composed  by  a  relation  and  the  concepts  or  objects  (idea  units)  that  it  interelates.  Together  the 
relations  and  idea  units  form  a  web  of  meaningfully  interconnected  ideas.  Sets  of  highly  interconnected  ideas 
form  schemas,  that  are  integratve  higher  level  units.  Schema  complexity  is  a  measure  of  the 
sophistication  of  factual  knowledge  used  in  the  performace  of  a  task. 

Imagistic  Knowledge,  refers  to  "knowing  what  something  looks  like".  Although  a  mental  image  has 
pictorial  elements  it  needs  not  to  be  an  exact  picture  of  the  thing  represented.  Instead,  the  mental  image  is 
an  arrangement  of  just  the  pictorial  elements  that  constitute  meaninglul  imagistic  properties  of  what  is 
being  represented  and  the  spatial  relationships  between  them.  A  useful  represenetation  of  imagistic 
knowledge  is  the  Pixel  Coordinate  Network,  proposed  by  Kossly  n  ( 1980).  Through  use  of  this 
representation  the  overall  image  can  be  mentally  transformed,  for  example  it  can  be  segmented  into 
subimages  of  meaningful  pictorial  elements  with  their  specific  spatial  coordinate  (a  'zoom  in'  kind  of 
operation).  These  pictorial  elements  can  be  used  for  imagistic  reasoning  by  means  of  such  mental 
transformations.  In  our  model,  imagistic  complexity  is  a  function  of  the  number  of  pictorial  elements  and 
density  of  the  relationships  between  them.  This  serves  as  an  indication  of  the  sophistication  of  imagistic 
knowledge  being  used. 

Procedural  knowledge  addresses  "knowing  haw  to  do  something".  It  can  be  represented  as  a  system  of 
production  rules  in  the  form  of  condition-action  pairs.  Simple  production  rules  are  clustered  in  plans  which 
integrate  highly  interconnected  actions.  Having  procedural  knowledge  accounts  for  more  effortless  free- 
flowing  performance,  in  the  execusion  of  a  task.  The  level  of  sophistication  of  procedural  knowledge  is 
then  indicated  by  plan  complexity  which  is  a  function  of:  (a)  the  number  of  actions  that  are  clustered  in  a 
plan  and  (b)  the  number  of  discriminating  conditions  to  which  the  plan  is  sensitive  (the  number  of 
alternative  procedures  for  the  same  goal).  In  addition,  the  distribution  of  tasks  among  workers,  by  means  of 
shared  procedures,  provides  insightful  descriptions  of  the  underlying  work  organization. 

Mechanism  Knowledge  is  represented  as  a  mental  model.  It  involves  knowing  "why  something 
works".  (Stevens,  1983)  It  therefore  refers  to  an  understanding  of  the  underlying  structure  of  a  system  that 
allows  for  the  procedures  used  to  perform  a  task  to  have  the  desired  outcome.  While  procedural  knowledge 
allows  smooth  performance,  a  "mental  model"  provides  easier  derivation  of  new  procedures  and  more 
complete  diagnosis  of  problems  that  might  arise.  This  knowledge  is  therefore  important  for  tasks  that  vary 
a  great  amount  and  often  involve  dealing  with  novel  situations.  Mental  model  complexity  (a  function  of 
the  number  of  system  components,  the  number  of  possible  states  of  the  mechanism  and  the  number  of 
transitions  between  these  states)  as  well  as  mental  model  flexibility  (the  ability  to  represent  the  system  in 
alternative  ways  relevant  to  particular  problems)  are  used  as  indicators  of  the  sophistication  of  the 
mechanism  knowledge  involved  in  the  performance  of  a  task. 

We  will  refer  to  the  four  representations  as  FIPM  in  the  rest  of  this  paper. 
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Our  data  collection  is  based  on  a  combination  of  methods  which  stem  from  efforts  in  anthropology  and 
cognitive  science.  The  methods  which  were  primarily  used  are  as  follows: 

On-Site  Observation 

For  on-site  observation  we  draw  upon  the  ethnographic  methodological  tradition.  Ethnographic  field 
methods  are  particularly  suitable  for  documenting  knowledge  as  situated  (Brown  et  a!.,  1989)  in  a  rich  social 
context.  We  used  observation  to  capture  various  aspects  of  the  work  context  -  physical  space,  work  objects, 
social  scene,  actors  and  actions,  time  sequencing  etc.  In  addition  we  entered  the  field  with  questions  based  on 
FIPM  regarding  observable  clues  as  to  the  types  of  knowledge  that  the  worker  utilizes.  For  example, 
smooth  execution  of  a  task  was  indicative  of  possible  existence  of  procedural  knowledge.  Our  fieldnotes 
also  include  verbal  reports  by  the  workers  about  their  various  tasks. 

Intensive  Interviewing 

A.  variety  of  interviewing  techniques  were  used  including  storytelling,  directed  questioning  and  critical 
incident  probes.  In  sequencing  the  interview  techniques,  it  was  important  that  the  storytelling  precede  all 
other  kinds  of  interviewing,  which  is  in  contrast  to  traditional  task  analysis  protocols.  We  have  adopted 
such  a  system  for  a  number  of  reasons:  (a)  when  open  ended  probes  follow  formal  accounts  of  the  job, 
elicited  by  means  such  as  directed  questioning,  previous  answers  constrain  storytelling  (b)  by  capturing 
knowledge  in  context  (in  storytelling  data)  we  are  able  to  look  at  the  FIPM  knowledge  representations  as 
situated  in  rich  contextual  information  including  time  sequencing,  social  interactions  and  affect.  Having 
established  this  general  context  of  knowledge  we  can  direct  our  inquiry  towards  specific  knowledge 
representations,  without  the  danger  that  our  data  will  be  overly  fragmented  and  (c)  it  provides  "anchoring" 
which  yields  a  more  natural  account  of  the  type  of  information  that  we  are  looking  for  than  does  answering 
abstract  questions.  The  subsequent  question -answer  processes  are  then  facilitated  by  the  fact  that  the  worker 
and  the  researcher  have  established  a  concrete  common  ground  of  understanding. 

Task  Related  Queries 

The  line  of  directed  questioning  is  guided  by  the  FIPM  model  as  the  questions  probe  for  worker's  use  of 
particular  knowledge  representations.  A  primary  tool  used  during  this  section  of  the  protocol  was  that  of 
the  task  matrix,  a  form  which  organizes  workers'  tasks  in  terms  of  the  types  of  knowledge  needed  to 
perform  those  tasks.  Information  about  this  knowledge  was  elicited  through  verbal  probes  designed  to 
capture  the  facts,  images,  procedures  and  mental  models  required  in  the  performance  of  each  specific  task. 

Critical  Incident  Probes 

Critical  incident  probes  are  also  FIPM  based  and  are  designed  (a)  to  elicit  alternative  procedures  to  a  goal  and 
(b)  to  evoke  mental  models  that  may  deal  with  atypical  situations,  in  this  case  the  FIPM  notation  served  as 
a  guide  to  elicit  knowledge  that  is  critical  for  efficient  performance,  even  if  it  is  not  spontaneously  evoked 
by  the  worker  during  the  short  period  of  data  collection. 

We  addressed  issues  of  reliability  and  validity,  thus,  through  these  efforts  to  triangulate  research  approaches. 
Through  the  examination  of  sites  with  such  a  varied  assortment  of  tools,  we  were  able  to  confirm  our 
findings  by  looking  for  consistencies  across  data  types. 

After  obtaining  field  notes  and  interview  data  from  each  site,  researchers  coded  the  notes  into  the  categories 
of  facts,  images,  procedures  and  mental  models.  In  addition,  the  category  "other"  was  used  to  hold  data 
which  did  not  fit  into  any  of  the  above  categories.  This  data  in  the  "other"  category  was  later  partitioned 
into  new  areas  such  as  goals,  affect,  social  interactions,  and  technology  integration.  In  cases  where  a  given 
piece  of  information  pertained  to  multiple  knowledge  representations,  it  was  coded  to  include  all  of  them^ 
Each  piece  of  data  was  then  "tagged"  with  the  worker's  name,  the  observer's/interviewers  name,  the  specific 
task  to  which  the  data  related,  and  the  portion  of  the  protocol  from  which  the  data  stemmed. 

This  method  of  coding  allowed  researchers  to  look  at  the  data  both  by  way  of  knowledge  needed  for  specific 
tasks  and  by  knowledge  as  it  is  organized  independent  of  tasks.  The  structure  of  our  data  thus  allows  for 
flexible  approaches  to  indexing  patterns  that  exist  across  tasks,  across  jobs,  and  across  workers.  Each 
researcher  examined  for  trends  in  the  data,  asking  questions  about:  what  types  of  knowledge  are  needed  to 
perform  each  specific  task,  which  types  of  knowledge  representations  are  present  in  numerous  tasks 


distributed  throughout  the  workers'  responsibilities,  which  tasks  constitute  larger  patterns  in  the 
organization  of  a  given  job,  and  how  do  different  workers  characterize  their  jobs  in  terms  of  different 
knowledge  representations. 

Thus  ourFIPM  model  is  guiding,  but  not  constraining,  our  data  collection.  Targeted  questions  and  task 
matrix  related  queries  were  used  to  ensure  that  the  main  tasks  of  the  job  are  recorded  and  that  their  specifics 
are  coded  in  terms  of  categories  recognizable  to  researchers,  as  well  as  in  terms  of  FIPM. 

Preliminary  Findings 

The  protocol  has  been  performed  at  14  sites  in  the  New  York,  Los  Angeles  and  Pulo  Alto  areas,  by 
ourselves  and  our  colleagues  at  the  Rand  Corporation.  For  the  purposes  of  this  paper  we  will  consider  six 
of  the  sites  at  which  the  cognitive  task  analysis  protocol  was  enacted  and  the  resulting  data  was  analyzed. 

site  1  -  x-ray  technicians  in  a  hospital 

site  2  -  records  center  clerks  in  a  law  firm 

site  3  -  customer  service  department  in  a  paper  company 

site  4  -  order  entry  department  in  a  clothing  manufacturer 

site  S  -  aides  in  a  home  health  agency 

site  6  -  copying  center  in  a  law  firm 

It  is  useful  to  frame  the  evaluation  of  our  work  to  this  point  with  respect  to  three  questions  that  address  our 
framework  and  methodology:  (a)  Is  the  FIPM  framework  generic  enough  to  base  a  domain  independent 
methodology  on?  (b)  are  the  distinctions  that  the  proposed  formalisms  make  meaningful  with  respect  to  the 
knowledge  that  workers  have  about  their  jobs?  and  (c)  are  the  aspects  of  job  related  knowledge  that  our 
methodology  captures  significantly  more  informative  than  traditional  task  analysis? 

We  have  found  the  FIPM  model  of  knowledge  representations  to  be  broad  enough  for  the  development  of  a 
domain  independent  method  of  cognitive  task  analysis.  Our  methodology  was  successful  in  eliciting  rich 
and  meaningful  data  across  cases  in  which  the  nature  of  the  job  and  the  organizational  structure  within 
which  the  job  was  performed  varied  considerably.  Each  of  the  four  representations  could  accommodate  a 
variety  of  knowledge  content  (knowledge  about  tasks,  knowledge  about  the  social  context,  etc.).  For 
example,  depending  on  the  specific  nature  of  each  job,  mental  model  knowledge  probes  were  successful  in 
eliciting  mechanism  knowledge,  organizational  system  knowledge  and  also  models  of  social  interaction. 

The  formalisms  proposed  by  the  FIPM  model  allowed  us  to  capture  differences  between  jobs  in  terms  of 
cognitive  demands.  Depending  on  the  nature  of  the  job,  different  types  of  knowledge  appeared  predominant. 

In  the  hospital  setting  in  which  x-ray  technicians  were  looked  at  (site  1),  for  example,  we  were  able  to 
elicit  detailed  imagistic  knowledge  regarding  body  marks  and  angulation  which  is  used  to  correctly  position 
the  patient  and  the  equipment.  Imagistic  knowledge  was  also  critical  to  the  task  of  successfully  evaluating 
the  diagnostic  quality  of  x-ray  films.  Together  with  being  able  to  characterize  the  x-ray  technician  job  as 
heavily  drawing  upon  imagery,  which  in  itself  may  seem  like  a  trivial  finding,  we  were  able  to  elicit  visual 
reasoning,  and  to  document  how  images  are  tied  to  (he  procedures  used  to  execute  the  tasks.  The  following 
examples  from  the  data  illustrate  this  point: 

^GU,  task  list,/:  "They  say  that  we  have  x-ray  vision  because  we  work  in  x-ray.  Whenever  I  have  a  case 
on  the  body  or  the  trunk,  we  don't  know  where  your  stomach  is,  1  don't  know  where  your  liver  is,  where  the 
vertebrae  are,  I  have  an  idea,  but  we  use  body  landmarks.  Depending  on  your  body  landmarks,  because  your 
elbow,  on  everybody,  you  let  your  arm  drop  to  the  side  of  your  body,  your  stomach  is  at  the  level  of  your 
elbow,  your  gall  bladder  is  at  the  level  of  your  elbow,  your  gall  bladder  is  at  the  level  of  your  kidneys 
which  is  about  LI.  If  you  just  close  your  eyes  and  you  just  picture  it,  if  I  just  bring  it  down  I'll  get  this 
part  of  the  body  that  1  need.  I  do  that,  it  gives  me  a  better  picture. " 

[GU,  Task-list,  pp.20-21,  supervising  students,  #  M,l,P]  E:  What  would  be  a  kind  of  problem  a 
student  would  come  up  with  ? 

G:  A  student?  A  kid,  they  take  a  hand  x-ray,  right?  You  do  it  like  this,  then  like  this.  A  kid's  hand's  in  a 
cast  and  they  want  to  straighten  the  figures  out,  but  they  can't  because  it's  in  a  cast.  So  it's  like  how  do  1 
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do  it?  Put  his  hand  down  and  shoot  it.  His  elbow  is  locked,  he's  supposed  to  do  an  AP  elbow  like  this, 
right,  what  you're  supposed  to  do  is  you're  supposed  to  put  it  like  this,  shoot  it  like  that,  and  you're 
supposed  to  put  it  down,  shoot  like  that.  What  1  do  is  1  put  it  on  the  V  and  I  shoot  straight.  They're 
confused,  it  gives  you  the  same  film,  just  a  different  way. 

With  respect  to  the  last  example,  it  is  important  to  note  that  although  the  overt  procedures  used  by  the 
technicians  are  standard  each  time,  for  any  given  x-ray  there  is  specialized  knowledge  which  informs  and 
alters  the  technicians'  procedures.  The  alternative  procedures  which  experts  have  at  their  disposal  involve 
considerably  more  than  procedural  knowledge.  Cognitive  task  analysis  has  allowed  us  to  elicit  information 
about  the  facts,  images  and  mental  models  which  inform  these  procedures.  Our  ability  to  gather  this  kind 
of  data  speaks  to  the  strengths  of  the  FIPM  framework,  especially  in  comparison  to  other  methods  of  task 
analysis. 

The  records  clerk  job  (site  2)  in  a  law  firm  record  center  provides  anothe;1  example  which  relates  to  this 
point.  The  record  center  job  is  highly  procedural  and  all  of  its  main  tasks  can  be  broken  down  into  a  series 
of  steps:  indexing,  filing,  binding,  searching  etc.  Varying  facts  are  continuously  plugged  in  the  procedures 
and  specify  alternative  routes  for  execution.  For  example,  the  document's  age  indicates  whether  it  should  be 
searched  in  the  on-line  system  or  in  the  warehouse  indexes  and  the  type  of  law  determines  how  documents 
will  be  bound.  Given  these  characteristics  this  job,  together  with  sites  3,  4  and  5,  can  be  said  to  belong  to 
a  broader  category  of  jobs  related  primarily  to  information  processing. 

The  sophistication  and  volume  of  the  record  keeping  aspects  of  this  job,  however,  differentiates  it  from  the 
other  information  processing  sites.  Through  applying  cognitive  task  analysis  we  were  able  to  discern  the 
particular  cognitive  functions  demanded  of  these  employees  and  recognize  thinking  skills  that  benefit 
performance  in  the  position.  The  file  clerks  deal  with  a  vast  and  complicated  body  of  facts  with  rigorous 
schemas  for  coding  and  filing.  They  are  intimately  familiar  with  the  files  to  which  they  are  assigned  and 
regularly  recognize  information  such  as  the  names  and  numbers  of  clients  and  cases  from  memory.  The 
clerks  are  also  familiar  with  types  of  law  and  types  of  legal  documents,  in  addition  to  knowing  relevant 
information  regarding  other  company  employees. 

Factual  information  is  at  the  core  of  ?11  the  procedures  which  records  center  clerks  perform.  Examples  of 
tasks  they  are  responsible  for  include  filling  a  request  form  by  phone  and  receiving  one  over  the  printer 
which  in  turn  would  initiate  a  search.  New  knowledge  is  generated  by  using  these  facts  around  requests 
against  their  filing  knowledge  and  experience.  Problem  solving  often  revolves  around  successfully 
completing  a  request  where  there  is  not  sufficient  information  about  the  document  requested.  This  usually 
involves  (a)  making  inferences  from  the  available  information  to  narrow  down  the  query  (b)  doing  searches 
using  the  computerized  and/or  physical  indexes  or  using  the  files  themselves,  or  (c)  attempting  to  obtain 
more  information  by  communicating  with  the  person  requesting  the  document  or  that  person  who  is 
holding  it.  Therefore  problem  solving  takes  the  form  of  querying  an  elaborate  network  of  facts,  both  within 
the  clerks  own  knowledge  and  within  that  knowledge  imbedded  in  the  firm's  database  systems. 

These  findings  suggest  that  indexing  knowledge  according  to  the  proposed  model  is  based  on  distinctions 
(among  facts,  procedures,  mental  models  and  images)  that  are  meaningful.  Cognitive  task  analysis  allowed 
us  to  gain  insight  into  the  rich  web  of  factual  information  record  clerks  employ  in  their  procedures. 
Through  this  analysis,  we  were  able  to  recognize  differences  in  the  cognitive  skills  needed  across  jobs  that 
would  on  the  surface  all  appear  to  serve  the  function  of  "information  processing". 

Apart  from  distinctions  pertaining  to  the  nature  of  the  job,  our  data  also  reflected  differences  in  the 
knowledge  that  front-line,  entry  level  workers  possess  as  a  result  of  the  organizational  structure  in  which 
they  perform  their  ,/i//„  This  analysis  provides  valuable  information  with  respect  to  the  issues  that  this 
study  was  designed  to  inform,  namely  the  changes  in  skill  demands  that  are  going  on  in  today's  workplace. 
In  moving  through  our  sites  of  study,  we  recognized  two  significant  trends  in  skill  demands  occurring 
within  our  sites  which  are  in  direct  contrast  to  one  another.  One  trend  is  that  of  transforming  traditionally 
low-skill  front-line  jobs,  such  as  clerical  jobs,  into  ones  which  provide  front-line  workers  with  increased 
responsibility  and  access  to  information.  The  contrasting  trend  is  placing  de-skilled  laborers  in  front-line 
positions.  We  see  this  occuring  in  fields  where  front-line  positions  have  traditionally  been  filled  with 
highly  skilled  workers.  We  found  cognitive  task  analysis  to  be  an  adept  tool  which  enabled  us  to  recognize 
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significant  distinctions  in  the  effects  of  these  organizational  decisions  on  worker's  knowledge  and 
performance. 

Site  3,  the  customer  service  department  of  a  large  paper  company,  provides  an  excellent  example  of  an 
organization  which  has  attempted  to  provide  entry-level,  front-line  workers  with  increased  access  to 
information  and  responsibility.  Three  years  before  we  examined  the  firm,  it  had  gone  through  a  structural 
re-organization  which  moved  traditionally  segmented  processes  such  as  order  entry,  customer  service,  and 
billing,  into  "high-performance  work  teams"  that  were  responsible  for  "all  aspects  of  the  revenue  cycle". 
The  teams  were  made  up  of  typically  five  to  seven  employees  who  shared  responsibility  for  meeting 
performance  goals.  Within  each  team,  workers  rotated  procedures  on  a  bi-monthly  basis  in  order  to  gain 
experience  in  a  variety  of  work  related  skills.  Team  members  worked  closely  with  one  another,  continually 
asking  questions  and  exchanging  pertinent  information  about  the  organization,  about  their  customers,  and 
about  social  activities-  for  example  their  daily  and  weekly  schedules.  This  team  based  environment  allowed 
knowledge  to  be  exchanged  very  easily,  in  a  manner  which  accommodated  the  workers'  immediate  needs. 
We  found  this  organization  made  unique  use  of  the  workers'  knowledge,  and  allowed  them  to  continually  add 
to  this  knowledge  set. 

As  was  argued  earlier,  data  about  job  related  knowledge  that  is  not  apparent  in  performance,  can  prove  to  be 
highly  informative  about  the  nature  of  a  given  position.  Site  4,  the  data  entry  department  of  a  clothing 
manufacturing  company,  serves  as  another  example  in  this  argument.  Data  entry  is  a  job  that  if  it  were  to 
be  described  as  a  list  of  observable  behaviors  would  appear  to  be  a  series  of  mechanistic  actions.  And  in 
fact,  in  the  cognitive  aspects  of  the  job  as  described  in  our  FIPM  schema,  procedural  knowledge  is 
predominant.  However,  we  were  also  able  to  elicit  factual  knowledge  about  the  company  that  doesn't  appear 
to  be  necessary  to  perform  this  particular  job,  as  well  as  a  systems  understanding  of  the  company's 
organization  and  the  flow  of  information  between  different  parts  of  the  company  (which  we  classify  as  being 
mental  model  knowledge).  In  addition,  open-ended  questions  aimed  at  capturing  cognitions  about  the  social 
context  also  elicited  informal  knowledge  (procedural)  about  efficiently  communicating  with  other 
departments  of  the  company. 

Workers  in  site  3  made  an  interesting  contrast  against  those  in  site  4.  While  the  customer  service 
employees  were  responsible  for  the  same  order  entry  duties  as  the  workers  in  site  4,  the  customer  service 
activities  were  done  in  the  context  of  the  "entire  revenue  cycle"  which  the  team  was  responsible  for. 
Through  rotating  tasks  related  to  all  aspects  of  this  cycle,  workers  attained  rich  mental  models  about  the 
organizational  systems  in  place.  And  by  being  organized  in  such  a  way  that  this  knowledge  could  be 
transmitted  easily,  the  entire  team  benefited  from  individual  worker's  knowledge  gains.  While  in  both  cases 
we  elicited  evidence  of  systemic  knowledge,  we  saw  that  the  customer  service  workers  were  asked  to  use 
this  mental  model  knowledge  continuously  in  order  to  manage  tasks  surrounding  the  revenue  cycle.  Their 
job  was  described  as  being  "multi-functional",  and  all  of  these  various  functions  were  concerned  with  this 
cycle.  The  data  entry  employees  in  site  4,  while  having  some  mental  model  knowledge  surrounding  their 
procedures,  were  generally  not  asked  to  use  this  knowledge  in  the  course  of  their  work. 

The  difference  between  these  two  organizational  structures  and  the  way  in  which  w  rkers  knowledge  is 
utilized  is  marked  and  appears  to  account  for  striking  differences  in  performance.  The  customer  service 
department  at  site  4  is  able  to  function  virtually  without  supervision,  and  is  able  to  experiment  and  find 
solutions  to  problems  in  a  way  that  the  workers  in  the  other  site  did  not  approach.  By  organizing 
responsibilities  in  this  manner,  workers  were  better  able  to  troubleshoot  and  make  high-level  decisions 
about  specific  cases.  When  asked  to  describe  workplace  success  stories,  these  employees  generally  answered 
with  cases  relating  to  a  high-order  change  they  brought  about  in  their  work  processes.  This  is  in  contrast  to 
the  workers  in  site  4,  whose  success  stories  related  to  common  procedural  problems  which  they  were  able  to 
overcome. 

Additionally,  by  organizing  the  worker's  responsibilities  in  terms  of  the  whole  revenue  cycle,  workers  were 
able  to  use  factual  information  pertaining  to  their  responsibilities  in  multiple  procedures.  For  example,  the 
customer  service  employees  were  responsible  for  procedures  such  as  handling  orders,  entering  data, 
coordinating  with  trucking  and  mills,  collecting  payment,  and  resolving  deductions.  Within  this  group  of 
procedures,  we  saw  a  tremendous  overlap  in  terms  of  the  factual  information  needed  to  perform  these 
procedures.  Examples  of  this  factual  information  include,  information  about  the  customer,  price  of 
products,  warehouse  location,  and  shipping  date.  This  organization  of  facts  and  procedures  is  in  contrast 
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with  that  of  the  clothing  manufacturer,  as  well  as  with  most  traditional  organizational  efforts.  Tracing  the 
path  of  facts  and  procedures  within  the  organization  not  only  lends  evidence  for  the  differences  between  these 
particular  sites,  but  also  points  to  cognitive  task  analysis  as  a  tool  that  can  serve  to  evaluate  organizational 
structures  in  terms  of  these  differences. 

A  parallel  contrast  which  also  looks  at  skills  demanded  of  entry-level,  front  line  workers  concerns  sites  1 
anH.  5.  In  case  study  1,  we  looked  at  x-ray  technicians  working  in  a  hospital  setting.  Site  5  involves 
home  health  aides  who  provide  health  care  in  the  greater  Los  Angeles  area. 

The  primary  responsibilities  of  the  home  health  aides  in  case  study  5  revolved  around  driving  to  patients 
homes,  collecting  information  about  the  patients'  well-being,  and  recording  this  information  in  a  uniform 
manner.  Aides  were  also  responsible  for  bathing  the  patient,  in  addition  to  several  other  incidental  chores. 
In  cases  when  aides  discovered  abnormal  occurrences,  they  would  confer  (via  telephone)  with  their 
supervisors  about  the  case,  and  proceed  to  follow  their  supervisors'  instructions.  The  job  requires  very  little 
in  the  way  of  imagistic,  factual,  or  mental  mode!  type  knowledge.  The  procedural  knowledge  needed  by 
these  workers  generally  revolved  around  their  asking  questions,  bathing  patients,  recording  information,  and 
scheduling  visits.  The  agency  which  employs  these  aides  also  employs  LVNs,  RNs,  and  medical  doctors. 
It  is  these  more  advanced  employees  who  take  on  diagnostic  health  care  duties. 

Our  analysis  of  case  study  1  showed  that  the  x-ray  technicians  utilized  comparatively  more  varied  and 
involved  knowledge  in  the  course  of  their  duties.  Here,  we  saw  workers  incorporate  involved  factual, 
imagistic,  procedural  and  mental  model  knowledge  in  areas  as  diverse  as  human  anatomy,  x-ray  equipment 
functionality,  and  radiology.  This  knowledge  allowed  workers  to  obtain  x-rays  even  in  cases  where  the 
health  status  of  the  patient  was  unusual,  or  the  circumstances  behind  the  x-ray  were  out  of  the  ordinary.  In 
addition,  the  x-ray  technicians'  knowledge  went  clearly  beyond  proper  use  of  the  equipment  and  patient 
positioning,  into  medical  knowledge  about  which  types  of  medical  procedures  require  particular  types  of  x- 
rays.  This  knowledge  allowed  technicians  to  continually  reassess  their  methods  according  to  their  given 
situation. 

The  x-ray  technician's  job  is  presently  a  high  skill  entry  level  job  that  requires  state  licensing  and 
certification.  While  we  were  on  site,  the  technicians  repeatedly  voiced  concern  about  a  trend  in  health  care 
reform  which  would  eliminate  this  licensing  requirement.  Workers  believed  that  the  training  which 
accompanied  licensing  was  a  mandatory  prerequisite  for  safe  and  intelligent  operation  of  x-ray  equipment, 
and  was  critical  to  their  patients'  health  and  well-being.  This  same  trend  appeared  to  have  already  occurred 
in  the  home  health  field,  where  RNs  and  LVNs  were  increasingly  being  replaced  by  aides.  Although  in 
most  cases,  surface  level  observations  of  tasks  which  these  positions  are  responsible  for  appear  to  be 
straightforward,  there  are  many  procedures,  as  was  described  earlier,  which  are  significantly  informed  by  deep 
underlying  knowledge.  As  the  trend  to  de-skill  these  entry  level,  front-line  workers  continues,  there  is  a 
threat  that  the  de-skilled  workers  will  not  have  sufficient  knowledge  to  inform  decisions  in  an  adequate 
portion  of  the  cases  they  are  exposed  to. 

We  see,  then,  that  a  traditional  job  description  of  the  home  health  aide  and  the  x-ray  technician  positions 
would  yield  considerable  similarities.  On  the  surface,  both  work  to  provide  health  information  for  doctors 
who  would  then  go  on  to  analyze  that  information.  The  cognitive  task  analysis  approach  yields  stark 
differences  between  the  knowledge  needed  in  order  to  perform  the  two  positions,  as  well  as  in  the  worker's 
ability  to  adapt  and  work  through  critical  incidents. 

Indexing  knowledge  in  terms  of  facts,  images,  procedures  and  mental  models  has  thus  yielded  distinctions 
between  jobs  that  capitalize  on  a  major  insight  of  cognitive  science,  namely  that  forms  of  knowledge  can  be 
made  distinct  and  may  account  for  performance.  In  addition,  our  emphasis  on  ethnographic  and  unstructured 
interviewing  methods  provide  us  with  rich  interpretive  data.  Together  these  provide  a  means  by  which 
cognitive  task  analysis  may  be  performed  that  incorporates  the  rich  contextual  information  inherent  in  the 
workplace. 
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Summary  of  Conclusions,  and  Implications  for  Research  and  Development 


The  FIPM  model  of  cognitive  task  analysis  has  enabled  us  to  make  unique  distinctions  between  jobs  in 
terms  of  the  knowledge  required  to  perform  those  jobs.  We  feel  as  though  one  area  which  may  significantly 
benefit  from  these  sorts  of  findings  is  that  of  job  descriptions.  Were  jobs  to  be  described  not  in  terms  of 
overt  acts,  but  in  terms  of  cognitive  activities,  we  may  be  able  to  enact  a  system  which  better  matches 
cognitive  skills,  to  tasks  which  incorporate  those  skills.  Also,  by  placing  these  descriptions  in  situated 
accounts,  job  seekers  may  begin  to  have  access  to  materials  which  more  accurately  reflect  the  positions  they 
consider. 

Cognitive  task  analysis  has  also  exhibited  great  potential  for  informing  areas  that  surround  organizational 
structure.  In  each  of  these  cases  described,  and  in  all  of  the  sites  we  visited,  we  found  that  workers  had 
knowledge  about  their  organization  which  went  far  beyond  that  which  was  required  of  them  to  perform  their 
job.  This  knowledge  often  came  about  through  informal  processes  in  which  information  about  the 
organization  was  exchanged.  As  stated  in  the  section  contrasting  the  customer  service  group  (site  3)  with 
the  order  entry  department  (site  4),  workers  can  put  this  knowledge  to  effective  use  in  cases  where  the 
organizational  structure  permits  such  activity.  Cognitive  task  analysis,  then,  may  serve  as  a  tool  to  inform 
organizational  decisions  around  how  to  best  structure  workers  so  that  the  organization  makes  most  effective 
use  of  their  workers'  knowledge.  An  effective  structure  would  optimize  performance,  as  well  as  allow 
knowledge  to  be  transmitted  between  workers  in  a  productive  manner.  In  our  study,  we  have  found  that 
those  organizations  which  allow  workers  to  put  their  knowledge  to  full  use  and  those  which  are  structured 
in  such  a  way  that  informal  knowledge  is  communicated  between  workers  on  a  regular  basis,  demonstrate 
differences  in  both  attitude  and  performance. 

In  terms  of  methodology,  we  found  that  the  task  matrix  approach  to  querying  subjects  was  successful  in  its 
allowing  us  to  verify  our  field  notes,  reduce  jobs  into  a  group  of  meaningful  tasks,  and  in  its  ability  to 
describe  each  of  these  tasks  in  terms  of  the  knowledge  types  required  for  performance.  While  the  data  which 
came  out  of  the  task  matrix  was  largely  qualitative  (due  to  the  fact  that  each  subject  had  varying  ideas  about 
what  constituted  an  image,  fact,  etc.)  we  see  an  opportunity  to  further  hone  this  methodology  into  one 
which  could  be  analyzed  quantitatively.  As  we  are  better  able  to  develop  cues  which  are  more  definitively 
interpretable,  we  will  be  able  to  more  powerfully  recognize  correlations  in  the  way  people  think  about  their 
work.  A  major  goal  as  far  as  future  efforts  are  concerned,  then,  is  to  hone  our  methods  so  that  they  may 
better  correlate  with  the  type  of  data  they  are  expected  to  elicit. 

Another  important  aspect  of  this  research  concerns  the  ability  for  the  workplace  knowledge  to  be  represented 
and  used  to  inform  instruction  and  training  efforts.  To  the  degree  that  our  research  was  done  within  the 
workplace  and  attempted  to  characterize  events  in  the  actual  context  of  their  taking  place,  the  data  we  draw 
upon  in  analysis  is  specifically  situated  in  workplace  culture  and  attitudes.  Additionally,  this  data  is  well- 
structured  around  both  the  given  tasks  and  the  types  of  knowledge  which  are  required  for  performing  those 
tasks.  Cognitive  task  analysis,  thus,  is  a  potentially  viable  source  for  site-specific,  situation-based 
educational  materials  (Ferguson).  We  see  that  materials  of  this  nature  are  used  in  instructional  technology 
corporate  training  efforts,  as  well  as  in  elementary  schools  (Black  &  Liepolt) 

Through  examining  and  formalizing  the  knowledge  workers  need  in  order  to  perform  their  jobs,  we  also 
were  able  to  gain  a  unique  look  at  the  types  of  materials  used  in  supporting  these  cognitive  activities.  We 
feel  as  though  there  is  considerable  possibility  for  analytic  tools  such  as  these  to  be  used  in  examining  the 
phenomenon  of  shared  cognition.  Workers  in  site  6,  the  law  firm  copy  center,  for  example,  relied 
considerably  on  the  ability  of  their  machines  to  provide  feedback  and  guidance.  We  witnessed  cases  in 
which  employees  were  able  to  use  knowledge  about  this  interface  to  "trick"  the  machine  into  performing 
tasks  other  than  what  the  interface  would  otherwise  lead  an  observer  to  believe.  As  these  relationships 
between  technology  and  workers  grow  more  prevalent  and  complex,  new  tools  will  need  to  be  developed 
which  analyze  these  relationships,  and  aid  workers  in  performance. 

In  summary,  we  have  established  that  our  model  of  knowledge  representation  is  effective  in  capturing  rich 
knowledge  content  both  in  relation  to  the  performance  of  specific  tasks  and  in  relation  to  the  general  work 
setting.  Future  efforts  in  this  area  may  have  significant  effects  on  areas  such  as  job  classification,  industrial 
organization,  instruction  and  technological  implementation. 
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A  "Layers  of  Negotiation  Model"  for  Designing  Constructivist  Learning  Materials 

Many  authors  have  questioned  whether  traditional  instructional  design  models  are  suitable  for 
designing  constructivist  learning  materials  (see,  for  example,  the  May  and  September  1991  issues  of 
Rducational  Technology).  Although  Winn  (1991)  questions  whether  "traditional  procedures  and 
assumptions  of  instructional  design"  can  be  applied  to  designing  materials  for  constructivist  learning 
environments,  he  admits  that "...  while  instructional  design  by  constructivists  may  seem  to  be  a 
contradiction  in  terms,  there  is  still  a  lot  of  designing  to  do."  (p.  39).  In  designing  materials  for  use  in  a 
constructivist  learning  environment,  instructional  designers  still  have  a  role  in  selecting  the  situations  that 
may  provide  a  stimulus  for  knowledge  construction  and  providing  features  that  support  students  and  teachers 
in  using  these  materials  (Young,  1993).  However,  Winn  (1991)  indicates  that  designing  materials  for 
constructivist  learning  environments  "requires  a  change  in  the  assumptions  about  how  people  learn  and 
about  how  instructional  decisions  are  made  "(p.  38).  These  discussions  imply  that  the  processes  of 
instructional  design,  not  just  the  products,  must  be  revised  to  accommodate  constructivist  ideas  about 
learning. 

The  purpose  of  this  paper  is  to  discuss  the  process  of  designing  a  series  of  case-based  interactive 
videodiscs  to  be  used  within  a  constructivist  learning  environment.  The  cases  present  two  elementary 
teachers  and  their  students  as  they  progress  through  science  lessons  based  on  the  conceptual  change  model  of 
science  teaching  (Cosgrove  &  Osborne,  1985).  The  two  hours  of  videodisc  footage  were  condensed  from 
approximately  60  hours  of  videotape  documenting  actual  classroom  events  occurring  during  2  weeks  of 
science  instruction  in  two  classrooms.  The  three  cased  include  a  1-week  lesson  on  levers  (5th  grade),  a  1- 
week  lesson  on  inclined  planes  (5th  grade),  and  a  2-week  lesson  on  seeds  and  eggs  (1st  grade).  The  discs 
illustrate  a  variety  of  on-location  classroom  events:  student  investigations,  small  group  interactions, 
student  record  keeping,  large  group  discussions,  and  demonstrations.  Dual  audio  tracks  allow  the  viewers  to 
hear  either  the  classroom  events  or  the  teacher's  reflections  on  the  classroom  events.  This  set  of  video-based 
cases  provide  a  context  for  preservice  elementary  science  teachers  to  engage  in  reflective  thinking.  Using 
these  materials,  students  have  the  opportunity  to  seek  answers  to  questions  that  evolve  from  their 
interactions  with  the  materials,  explore  the  stages  in  conceptual  change  science  teaching,  and  gain 
experience  with  reflection  on  teaching. 

As  we  designed  these  materials,  we  were  guided  by  a  set  of  constructivist  assumptions  about 
learners  and  learning.  These  assumptions  had  implications  for  the  learning  materials  and.  we  believed,  for 
the  instructional  design  process.  We  assumed  that  traditional  instructional  design  models  would  not  provide 
adequate  guidance  in  designing  these  materials;  instead,  we  sought  to  design  our  materials  guided  by  our 
assumptions  about  teaching  and  learning,  and  once  the  materials  had  been  developed,  to  reflect  back  over  the 
design  process  and  compare  our  procedures  with  those  prescribed  by  traditional  models.  This  paper 
documents  that  process:  we  outline  the  assumptions  about  teaching  and  learning  that  guided  our  design 
efforts,  we  discuss  the  implications  of  these  beliefs  for  the  instructional  materials,  we  describe  our  design 
process,  and  finally,  we  derive  some  generalizations  about  the  differences  between  our  process  and  the 
procedures  recommended  by  traditional  instructional  Jesign  models  that  may  be  of  use  to  other  designers. 
Assumptions  about  learners  and  learning 

We  assume  that  learners  come  to  an  instructional  setting  with  a  wide  variety  of  background 
experiences  that  have  shaped  their  understandings,  beliefs,  and  values.  Thus,  the  meaning  that  a  learner 
derives  from  an  instructional  experience  may  not  be  the  same  as  another  learner's;  an  individual's  experience 
is  mediated  through  a  unique  filter  of  understanding,  beliefs  and  values  (Freyberg  &  Osborne,  1985). 
Learners  may  focus  on  different  inputs  or  all  may  attend  to  the  same.  data.  Similarly,  there  is  no  guarantee 
that  an  instructional  intervention  will  "work"  for  every  student;  student's  ideas  could  be  uninfluenced  or 
influenced  in  unanticipated  ways  by  the  intervention,  depending  on  the  learner's  action  on  the  experience 
(Gilbert,  Osborne,  &  Fenshem,  1982). 
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We  believe  that  learning  is  a  process  of  sense-making,  of  assimilating  new  information  within 
existing  knowledge  structures,  and  adjusting  prior  understandings  to  accommodate  new  information.  In 
order  to  facilitate  the  assimilation  and  accommodation  of  new  experiences,  we  believe  that  disequilibrium  is 
essential.  Learners  must  be  dissatisfied  with  their  existing  knowledge  and  beliefs  in  order  for  learning  to 
take  place  (Posner,  Strike,  Hewson,  &  Gertzog,  1982). 

In  addition,  we  believe  that  learning  is  a  social  enterprise.  Learners  can  become  dissatisfied  with 
their  existing  knowledge  when  they  compare  their  ideas  with  ideas  of  others,  both  experts  and  peers 
(Solomon,  1989).  Thus,  through  this  sharing  of  knowledge,  learners  may  find  new  ideas  that  are 
intelligible,  plausible,  and  useful  alternatives.  Through  social  interactions,  as  well  as  through  action  on 
objects,  learners  make  sense  of  the  world. 
Implications  for  learning  materials 

Grounded  in  our  beliefs  about  the  nature  of  learners  and  learning,  are  implications  for  the  learning 
materials.  We  recognize  that  learners  may  come  to  the  instructional  setting  with  varied  background 
knowledge,  beliefs,  and  attitudes;  therefore  the  materials  must  be  capable  of  supporting  a  variety  of  learners 
possessing  a  variety  of  perspectives  as  they  attempt  to  create  meaning  from  the  instructional  materials. 
Similarly,  meaning  is  not  inherent  in  instructional  material;  instead  meaning  is  created  by  learners  as  they 
interact  with  the  materials,  with  the  teacher,  and  with  other  learners. 

From  her  synthesis  of  recommendations  from  numerous  researchers  "attempting  to  articulate 
constructivist  theory",  Driscoll  (1994,  p.  365)  identifies  five  conditions  that  should  be  incorporated  within 
constructivist  learning  environments.  She  suggests  that  constructivist  learning  environments  should: 

1 .  Provide  complex  learning  environments  that  incorporate  authentic  activity. 

2.  Provide  for  social  negotiation  as  an  integral  part  of  learning  to  allow  insights  to  emerge 

through  the  group  process  that  may  not  come  about  otherwise. 

3.  Juxtapose  instructional  content  and  include  access  to  multiple  modes  of  representation  to  allow 

learners  to  examine  materials  from  multiple  perspectives. 

4.  Nurture  reflexivity,  or  awareness  of  one's  own  thinking  and  learning  processes. 

5.  Emphasize  student  centered  instruction,  where  students  are  actively  involved  in  determining 

their  own  learning  needs  and  how  those  needs  can  be  met. 
As  the  following  discussion  illustrates,  our  materials  incorporate  each  of  these  recommendations. 

1 .  Provide  complex  learning  environments  that  incorporate  authentic  activity.  We  recognize  that 

classrooms  are  complex  environments  where  many  events  occur  simultaneously.  Using  interactive 
videodiscs  of  actual  classroom  lessons,  the  preservice  teacher  can  enter  a  classroom  virtual  world. 
Viewers  can  begin  to  identify  problematic  classroom  events  through  the  rich  visual  and  verbal  cues 
provided  via  videodisc.  Rather  than  provide  brief  vignettes  of  exemplary  teaching  (see,  for 
example,  Goldman  &  Barron,  1990),  we  choose  to  present  three  complete  units  of  science 
instruction.  The  viewers  witness  the  natural  progression  of  events  as  they  unfold  over  a  one  or 
two  week  time  period.  They  see  things  that  work  and  things  that  don't.  They  witness  the  teacher's 
daily  reflections  on  the  classroom  events,  her  frustrations,  her  pride,  and  her  careful  consideration 
of  alternatives  for  action.  In  this  way,  the  complexity  of  the  classroom  environment  is  maintained 
in  authentic  recordings  of  the  classroom  events. 

2.  Provide  for  social  negotiation  as  an  integral  part  of  learning  to  allow  insights  to  emerge  through  the 

group  process  that  may  not  come  about  otherwise.  As  these  cases  are  used  in  university  methods 
classes  to  coach  preservice  science  teachers  to  become  reflective  practitioners,  social  negotiation  is 
an  integral  part  of  the  teaching/learning  process.  A  series  of  written  and  oral  reflective  tasks 
cumulate  in  large  group  discussions  where  students  critically  analyze  various  classroom  episodes 
from  the  videodiscs.  As  they  examine  their  beliefs  and  values  about  teaching  science,  they  may 
find  that  their  theories  are  challenged  by  other  ideas  that  emerge  through  group  discussions  (Abell, 
Cennamo,  Anderson,  Bryan,  Campbell,  &  Hugg,  in  press).  Although  the  class  may  seldom  reach 
consensus,  each  student  builds  upon  his  or  her  own  personal  theories  based  on  these  experiences. 
In  this  way,  insights  about  teaching  and  learning  science  emerge  through  the  group  process  that 
may  not  occur  otherwise. 

3.  Juxtapose  instructional  content  and  include  access  to  multiple  modes  of  representation  to  allow 

learners  to  examine  materials  from  multiple  perspectives.  The  materials  provide  students  with 
access  to  multiple  modes  of  representing  the  classroom  events.  The  videodiscs  contain  dual  audio 
tracks  that  allow  the  users  to  either  hear  the  naturally  occurring  classroom  audio  or  the  teachers' 
reflections  on  the  classroom  events  illustrated  in  the  video.  The  videodiscs  are  accompanied  by  a 
HyperCard  stack  that  includes  student  products,  background  information  on  the  school,  additional 
information  on  the  teacher  and  students,  lesson  plans,  and  reference  materials.  As  students  explore 
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tlie  materials,  they  can  read  the  lesson  plan,  see  the  classroom  events,  listen  to  the  teacher's 
reflections  on  the  classroom  events,  examine  student  products,  read  more  about  the  students,  learn 
more  about  the  teaching  strategies  used  in  the  lesson,  or  access  several  other  frames  of  information 
related  to  a  single  classroom  event.  With  the  random  assess  possible  with  computer-controlled 
interactive  videodisc,  users  can  juxtapose  information  in  a  sequence  of  their  own  choosing  and 
examine  the  content  from  multiple  perspectives. 

4.  Nurture  reflexivitv.  or  awareness  of  one'  own  thinking  and  learning  processes.  Our  interactive  video 

cases  are  designed  to  encouraged  preservice  teachers  to  become  aware  of  their  own  thinking  and  learning 
proc  esses.  A  number  of  written  and  oral  reflective  tasks  have  been  developed  for  use  in  conjunction 
with  the  video  cases.  These  tasks  systematically  and  developmentally  attempt  to  help  preservice 
teachers:  1)  uncover  their  local  theories  of  science  teaching  and  learning;  2)  recognize  the  role  that 
personal  histories  of  science  learning  play  in  their  local  theories;  3)  realize  that  classrooms  are  complex 
and  that  classroom  observers  focus  on  various  attributes;  4)  frame  issues  that  arise  during  science 
instruction;  and  5)  reassess  their  own  theories  in  light  of  their  own  and  others'  classroom  practice. 
Thus  the  video  cases  encourage  reflection  about  teaching  and  learning. 

5.  Emphasize  student  centered  instruction,  where  students  are  actively  involved  in  determining  their 

own  learning  needs  and  how  those  needs  can  be  met.  As  described  above,  the  materials  allow  users 
to  choose  several  modes  of  representation  in  order  to  meet  their  information  needs.  In  addition, 
several  lines  of  investigation  can  be  pursued.  These  materials  can  be  used  to  better  understand  the 
lesson  stages  in  conceptual  change  science  teaching  (Cosgrove  &  Osborne,  1985),  children's  ideas 
about  science,  the  science  content,  or  reflection  on  teaching.  The  materials  are  designed  to  be  used 
with  ease  in  large  groups,  in  small  groups,  or  individually.  Thus,  the  materials  provide  students 
with  the  flexibility  to  explore  in  ways  that  allow  them  to  determine  their  own  learning  needs,  to 
determine  how  those  needs  can  be  met,  and  to  pursue  information  to  meet  these  needs  within  the 
context  of  elementary  science  instruction. 
Implications  for  the  design  process 

In  designing  the  interactive  video  case  materials,  our  beliefs  about  the  nature  of  learning  also  had 
implications  for  the  instructional  design  process.  Just  as  constructivists  emphasize  the  process  of  learning, 
rather  than  merely  the  product,  we  believed  that  the  process  of  designing  these  materials,  not  just  the 
product,  should  be  consistent  with  constructivist  theory.  Not  only  should  the  materials  themselves  be 
developed  for  use  in  a  constructivist  learning  environment,  but  we  believed  that  the  materials  should  evolve 
though  a  process  of  construction  of  knowledge,  reflection,  and  social  interactions.  In  retrospect,  we  propose 
that  Driscoll's  (1994)  five  conditions  for  constructivist  learning  environments  can  be  transformed  to  apply 
to  the  process  of  designing  materials  from  a  constructivist  perspective.  We  suggest  the  instructional  design 
process  should  include  the  following  conditions: 

1 .  Embrace  the  complexity  of  the  design  process. 

2.  Provide  for  social  negotiations  as  an  integral  part  of  designing  the  materials. 

3.  Examine  information  relevant  to  the  design  of  the  instruction  at  multiple  times  from  multiple 

perspectives. 

4.  Nurture  reflexivity  in  the  design  process. 

5.  Emphasize  client-centered  design. 

Although  we  addressed  these  five  conditions  throughout  the  entire  instructional  design  process,  for 
simplicity,  we  will  illustrate  each  condition  as  we  describe  one  aspect  of  designing  and  developing  our 
materials.  After  recording  approximately  60  hours  of  videotaped  footage,  a  we  faced  a  major  task  in 
selecting  the  video  segments  to  be  included  within  two  hours  of  disc  space.  The  process  we  followed  in 
accomplishing  this  complex  task  is  described  below: 

1 .  Embrace  the  complexity  of  the  design  process.  Instructional  design  has  traditionally  been  a  prescriptive 
science  (Regeluth,  1983);  however,  we  found  that  attempting  to  simplify  the  design  process  and  apply 
strategies  that  had  worked  in  other  situations  had  the  potential  for  limiting  the  richness  of  our  product. 
Foremost  in  our  minds  was  to  create  "cases"  of  conceptual  change  science  teaching  in  elementary 
classrooms.  In  order  to  make  decisions  on  the  footage  to  be  included  on  our  disc,  the  project  team  had 
to  agree  upon  what  would  constitute  a  classroom  "case."  At  first  we  looked  for  prescriptions  for  this 
design  dilemma.  However,  our  cases  were  different  from  most  in  that  they  were  video-based,  rather 
than  print-based,  thus,  guidelines  for  constructing  cases  (e.g.,  Carter,  1992;  Shulman,  1992)  were 
insufficient.  Guidelines  pertinent  to  videodisc  cases  in  elementary  mathematics  (see  Goldman  & 
Barron,  1990}  provided  some  clues.  Yet,  the  nature  of  science  compared  to  mathematics  teaching 
presented  some  unique  needs.  In  addition,  we  knew  that  our  lessons  extended  over  several  days  and  that 
it  would  be  necessary  to  capture  the  instruction  over  this  extended  time.  Thus,  instead  of  relying  on 
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any  existing  set  of  prescriptions,  we  accepted  the  need  to  embrace  the  complexity  of  our  task  and  to 
create  our  own  process  for  selecting  video  segments. 
jgg°f"  gag  P^tMonS  «  an  integral, part  of  the  matenals.  After  reject, ng  a 

described  way -of  select ng  scenes  for  inclusion  on  our  disc,  we  engaged  in  extensive  negot.at.ons  to 
Eft Appropriate 'scenes  Our  selection  process  began  by  establishing  a  shared  perspective  among 
earn  memSs  (2  methods  course  instmctors,  1  graduate  assistant,  1  science  content  specialist,  and  1 
n"onaf  de  igner)  on  the  types  of  scenes  needed.  We  discussed  the  needs  of  our  parucular  aud.ence 
Indin mctor  relative  to  our  content:  the  need  to  illustrate  key  stages  in  the  lesson,  problem*  c 
clas  Sm  even[s  and  examples  of  the  teachers'  reflection  in  action.  We  decided  to  search  tape  logs  for 
scenes  Aat  wouW  represent  topics  we  discussed  in  our  classes,  that  exemplified  particular  teaching 
U-ategies ^  demonstrated  children's  conceptual  development,  and  that  illustrated  the  complete,  of 


3. 


ThZghout  this  process  each  team  member  had  to  negotiate  his  or  her  perspectives  with  other 
team  m^m£UtSince  each  of  us  came  to  the  project  with  different  experiences  and  experuse  we  needed 
\  m,7'V  .ltural  knowledge"  (Driscoll,  1994)  in  order  to  defend  our  selecuons.  Our  reasoning 

^d^S^JSX  classes,  what  we  believed  about  teaching  and  learning,  and  what  we 
lut le  science  content  involved.  Through  this  negotiation  of  shared  rnearungs  we  were 
aMeTo  create  a  richer  product.  Our  collaborative  problem  solving  became  a  key  component  ,n  the 

rtSrn1nrPiSpTi"P  relevant  to  the  dmm^^mp^  at  multiple  times  &»liipk 
E^SItprS^agues  (Spiro,  Feltovich,  Jacobson  &  Coulson,  1991)  state  that 
StSe  same  material,  at  different  times,  in  rearranged  contexts,  for  different  purposes,  and  from 
aSXSS  Aspect  ves  is  essential  for  attaining  the  goals  of  advanced  knowledge  acquisition 
t  m  W SenffiK  is  also  important  for  the  instructional  design  team  to  conunually  evisrt  the 
Le  ^tTor  ideas  at  different  times,  in  rearranged  contexts,  and  for  different  purposes  in  order  to  create 

riCh  M'we^dentfied^e^m^nte  S?  ou^taped  footage,  we  noted  how  they  could  be  used  to 
illus^e ™S  p    ideafTn  the  university  classroom.  We  found  that  we  needed  to ,  iev»,  our  footage 
seveS  times  After  our  initial  selection,  we  created  a  rough  edit  of  our  footage  that  was 
S£  Twice  as  long  as  our  finished  product.  We  reviewed  this  tape  extensively  to  determ  ne 
Xher^h continuity  of  the  lesson  was  maintained  and  to  identify  segments  that  could  be  shortened 
23£  mate A  seZd  rough  edit  was  necessary  before  we  made  our  final  edit  decisions.  We  found 
That  r  often  had  to  add  sections  to  maintain  the  continuity  of  the  lesson.  Using  our  second  rough 
edh  we  began  to  match  the  teachers'  reflective  segments  to  the  classroom  footage.  As  the  tape  was 
revLwed  S  Potential  users  of  the  product  (university  elementary  science  methods  'nstructors)^ and  the 
nSfonal  Egner,  each  individual  examined  it  from  his  or  her  perspective  as  influenced  by  our 
ore^s  SsSns  As  we  revisited  this  data  at  different  times,  in  various  contexts,  and  with 
C  purposed •„  mind,  we  maintained  the  richness  inherent  in  the  original  event,  yet  shortened  it 
tn  a  manaopable  time  frame  for  classroom  use.  ,      .         .  . 

i  ,Tl^^!!-tg  ^^f.^n  process.  When  reviewing  instructional  matenals,  instructional 

S  £ion  needed  on  the  disc.  It  was  critical  for  each  of  us  to  be  aware  of  the  reasoning  behind  our 
™tions  in  order  to  clearly  negotiate  the  selection  of  the  final  videotape  segments. 
5  EmSSSLMsn.  Most  designers  can  recall  situations  where  the  client  approved  an 
^^^^M^t^  final  product  was  delivered,  felt  that  it  wasn't  quite  what  tiiey  had 
n  Snd    Manv  c Sits  are  a  little  unsure  exactly  what  they  want  initially;  unlike  experienced 
desTners  E  unabk  to  envision  the  finished  product  until  they  see  it.  With  c hent-cen  ered 
£ifn  clients  a^  defined  as  representative  of  those  who  will  ultimately  use  the  matenals  for 
Sction  rather  than  those  who  fund  the  project.  These  clients,  in  col  aboration  with  the 
nsSon^uSg^r.  must  be  actively  involved  in  determining  what  their  needs  are  and  how  those 
T.T^be Satisfied  As  we  designed  our  materials,  those  who  would  eventually  use  the 
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best  be  satisfied.  Using  this  approach,  the  clients  were  able  to  refine  their  needs  as  the  project  evolved, 
to  clarify  needs  as  additional  data  became  available,  and  to  constantly  be  involved  in  decisions  at  each 
stage  in  the  design  negotiation  process. 
Analysis  of  the  Process 

Although  we  did  not  follow  a  traditional  model  of  instructional  design,  in  retrospect,  we  believe 
that  our  design  process  was  indeed  systematic.  Unlike  the  process  outlined  in  traditional  instructional 
design  models  (e.  g.  Dick  and  Carey,  1990;  see  Figure  1),  we  did  not  proceed  through  discrete  .stages  of 
analysis,  design,  development,  and  evaluation.  Consequently,  the  output  of  one  stage  in  the  model  did  not 
provide  the  input  for  the  next  stage.  However,  the  majority  of  issues  addressed  by  traditional  models  of 
instructional  design  were  addressed  as  the  design  and  development  of  the  case-based  interactive  video 
materials  evolved  through  the  construction  of  shared  meanings.  As  in  Tessmer  and  Wedman's  ( 1990) 
"Layers  of  Necessity  Model"  of  instructional  design,  the  design  of  our  materials  evolved  in  a  spiral,  layered 
fashion.  But  whereas  Tessmer  and  Wedman's  model  suggests  that  designers  select  a  layer  of  precision  based 
on  the  time  and  resources  available  to  the  designer,  in  our  "Layers  of  Negotiations  model",  we  suggest  that 
designers  proceed  to  deeper  levels  in  the  process  when  additional  data  becomes  available  or  relevant  to  the 
discussions.  When  designers  embrace  the  complexity  of  the  design  process  and  emphasize  client-centered 
design,  social  negotiations  become  an  integral  part  of  the  design  process.  As  designers  and  clients  negotiate 
shared  perspectives,  both  parties  are  required  to  become  reflexive  and  articulate  their  thought  processes. 
Through  this  process,  ideas  and  data  are  examined  from  multiple  perspectives. 


Figure  1 

Systems  Approach  Modei  for  Designing  Instruction 
(Dick  and  Carey,  1990) 
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Our  "Layers  of  Negotiation"  process  is  characterized  by  a  number  of  distinctions  when  compared  to 
traditional  instructional  design  models: 

Question  driven  rather  than  task  driven.  Rowland  (1992)  suggests  that  instructional  designers 
should  focus  on  asking  good  questions,  rather  than  simply  following  the  "steps"  prescribed  by  traditional 
instructional  design  models.  Using  the  Dick  and  Carey  (1990)  model  as  an  example,  we  found  that 
although  we  did  not  proceed  through  the  steps  in  a  linear  fashion,  we  addressed  most  of  the  questions  posed 
by  the  model  (see  Figure  2).  However,  the  answers  to  many  of  the  "questions"  posed  by  the  Dick  and 
Carey  model  of  instructional  design  were  very  different  from  those  suggested  by  Dick  and  Carey  (1990).  In 
addition,  as  the  design  and  development  of  the  case-based  interactive  video  materials  evolved  through  a 
process  of  reflection  and  social  negotiations  of  shared  meanings,  we  never  performed  a  formal  instructional 
analysis  or  developed  learning  objectives.  Instead,  we  performed  a  content  analysis  of  our  videotape  footage 
to  identify  patterns  of  events  and  clustered  them  for  access  by  the  learners.  Consistent  with  Jonassen's 
(199 1)  recommendations,  our  analysis  focused  on  considering  various  interpretations  of  the  content  and 
providing  the  "intellectual  tools  that  are  necessary  for  helping  learners  construct  knowledge"  (p.  12). 
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Figure  2 

Sample  Questions  Based  on  the  Stages  in  the 
The  Systematic  Design  of  Instruction 
(Dick  and  Carey,  1990) 

Identify  Instructional  Goals 

What  should  be  included  in  the  curriculum? 

Is  time  and  money  available  for  development  of  the  materials? 

How  should  the  needs  be  assessed? 

Who  are  the  learners? 

What  is  the  goal  of  the  learners? 

Can  the  learners  achieve  the  goal? 

Does  the  need  of  the  stakeholders  match  with  the  goal  of  instructional  materials? 
Are  there  sufficient  people  to  complete  this  material? 
Is  there  sufficient  time  to  complete  this  job? 

Are  these  goals  acceptable  to  those  who  must  approve  this  instructional  development  effort? 
Will  the  development  of  this  instruction  solve  the  problem  which  led  to  the  need  for  it? 
What  is  the  content  to  be  taught? 

Is  content  stable  enough  to  warrant  the  cost  of  development? 
Are  the  learners  available? 

Is  the  designer  experienced  in  content  or  instructional  design? 

Identify  Entry  Behaviors  and  Characteristics 

What  are  the  critical  entry  skills  required  of  students  before  the  instruction? 

What  is  the  learner's  general  ability  level? 

What  is  the  learner's  previous  experiences? 

What  is  the  learner's  expectations  about  instruction? 

How  will  the  learner's  characteristics  impact  the  design  of  instruction? 

Conduct  Instructional  Analysis 

What  are  the  goals  of  instruction? 
What  will  be  taught? 

What  is  the  learning  domain  of  the  target  skill? 

What  would  a  learner  be  doing  when  performing  the  goal? 

What  are  the  subordinate  skills  to  reach  the  goal? 

What  is  the  relationship  between  these  skills? 

What  skills  can  be  omitted  from  the  instruction? 

What  is  it  that  the  students  must  already  know  how  to  do,  the  absence  of  which  would  make  it 
impossible  to  learner  this  subordinate  skills? 

Write  Performance  Objectives 

What  is  it  that  the  learner  should  be  able  to  do  at  the  end  of  the  instruction? 

Under  what  conditions  should  the  learner  perform  the  target  tasks? 

What  are  the  skills  the  learner  has  to  perform? 

What  criteria  will  be  used  to  assess  the  performance  of  the  learner? 

What  is  expected  of  the  learners? 

What  are  the  objectives  for  the  entry  behaviors? 

Develop  Criterion-Referenced  Test  Items 

How  will  learning  be  assessed? 
What  will  be  tested? 
When  will  be  evaluated? 
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How  often  the  learners  be  evaluated? 

What  is  the  criterion  level  to  be  considered  as  mastery? 

What  kind  of  feedback  will  be  used? 

Do  the  test  items  match  with  objectives? 

By  what  methods  will  the  learning  be  assessed?  e.g.  paper-pencil,  observations,  etc. 
Do  the  test  items  match  the  behavior  and  the  conditions  specified  in  the  objectives? 
Do  the  test  items  provide  students  with  the  opportunity  to  meet  the  criteria  needed  to 

demonstrate  mastery  of  objectives? 
How  many  test  items  should  be  used? 
What  is  the  weaknesses/benefits  of  selected  type  of  tests? 
How  long  will  the  assessment  require? 
How  much  time  is  required  for  students  to  complete  the  test? 
What  will  the  testing  environment  be  like? 
What  is  the  probability  of  guessing  ihe  answers? 
Did  the  assessment  undergo  formative  evaluation? 
Who  would  administer  the  assessment  to  students? 

Develop  Instructional  Strategy 

How  will  the  information  be  chunked  (clustered)? 
How  will  the  information  be  sequenced? 
What  pre-instructional  strategies  will  be  used? 
How  will  learners  become  motivated? 

How  will  they  become  aware  of  the  necessary  prerequisite  skills? 

How  will  they  be  informed  of  the  objectives? 

How  will  the  information  be  presented? 

What  information  be  presented? 

How  will  examples  be  used? 

How  will  the  learners  participate  in  instruction? 

How  will  feedback  be  provided  to  the  learners? 

When  will  learning  be  assessed? 

How  long  would  it  take  the  learners  to  learn  the  skills? 

Develop  and  Select  Instructional  Materials 

What  are  the  instructional  strategies  to  be  used? 

Should  the  materials  be  for  individualized  or  group  instruction? 

Are  there  existing  materials  that  can  be  used? 

What  is  the  intended  delivery  mode  for  the  instruction? 

Which  media  will  be  used? 

How  and  when  will  the  assessments  be  administered? 

What  should  be  included  in  the  instructors  manual? 

What  is  the  role  of  instructor? 

How  should  the  instructor  use  the  materials? 

What  is  included  in  the  student  materials? 

How  should  the  students  use  the  materials? 

Develop  and  Conduct  Formative  Evaluation 

Does  the  content  expert  believe  the  content  of  the  materials  is  accurate? 
Does  the  content  expert  believe  the  content  of  the  material  is  current? 

Does  the  learning  specialist  believe  the  target  population  will  have  potential  problems  with  the 
materials? 

How  does  the  learning  specialist  think  the  prospective  learners  will  react  to  the  materials? 
What  do  the  target  learners  say  about  the  material? 

What  is  the  most  obvious  errors  in  the  instruction  when  used  by  the  target  learners? 
What  is  the  initial  reactions  of  the  target  learners  to  the  content? 
Can  learners  use  the  materials  without  interaction  with  the  instructor? 
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Are  the  changes  effective? 

Are  there  additional  errors  or  problems  with  the  materials? 

Do  the  target  learners  believe  the  instruction  is  interesting? 

Do  the  target  learners  understand  what  they  are  supposed  to  learn? 

Do  the  target  learners  believe  the  materials  are  directly  related  to  the  stated  objectives? 

Do  the  target  learners  believe  there  are  sufficient  practice  exercises  included? 

Do  the  target  learners  believe  the  practice  exercises  are  appropriate? 

Do  the  target  learners  believe  the  tests  really  measure  their  performance  on  the  objectives? 

Do  the  target  learners  believe  they  receive  sufficient  feedback  on  their  practice  exercises? 

Do  the  target  learners  believe  they  receive  sufficient  feedback  on  their  test  results? 

Do  the  target  learners  believe  the  enrichment  or  remedial  materials  were  satisfactory? 

Can  the  materials  be  used  effectively  in  the  intended  instructional  settings? 

Who  will  conduct  the  formative  evaluation? 

What  instruments  will  be  used? 

What  kind  of  data  will  be  collected? 

Are  the  materials  designed  efficient  and  effective? 

What  problems  do  the  materials  have  when  actually  used? 

What  is  the  time  needed  to  complete  various  components  of  the  instruction? 

Process-based  versus  procedure-based  design.  Whereas  traditional  instructional  design  models 
prescribe  a  set  of  procedures  to  be  followed  to  design  instruction,  we  found  our  emphasis  shifted  to  the 
process  of  decision-making  that  is  involved  in  designing  instruction.  With  a  client-centered  design  process 
that  emphasized  reflexivity,  the  act  of  decision-making  became  an  important  aspect  of  the  knowledge 
building  which  contributed  to  the  design  and  development  of  the  materials.  Just  as  learners  come  to  an 
instructional  experience  with  understandings,  beliefs  ,and  values  that  have  been  shaped  by  their  prior 
experiences,  members  of  the  instructional  design  team  also  begin  the  process  of  designing  instruction  with 
their  individual  sets  of  understandings,  values,  and  beliefs.  The  process  of  designing  instruction  included 
negotiating  a  set  of  shared  beliefs  that  guided  the  development  of  the  materials.  Initial  discussions  focused 
on  creating  a  shared  philosophy  of  learning,  identifying  the  type  of  knowledge  change  desired,  and 
determining  ways  to  assess  the  learners'  knowledge  development.  Examining  instructional  decisions  from 
the  multiple  perspectives  and  "cultural  knowledge"  of  the  individuals  in  the  design  team  enhanced  the 
possibilities  that  emerged  from  the  design  process. 

Spiral  cvcl£s_rather  than  discrete  stages.  Whereas  traditional  instructional  design  models  (e.g.  Dick 
and  Carey,  1990  )  include  discrete  stages  for  analysis,  design,  development,  and  evaluation  activities,  we 
addressed  the  questions  of  design  in  a  spiral  fashion,  progressing  through  a  series  of  steps  at  one  level,  then 
spiraling  back  and  adding  more  detail  within.  We  initially  made  decisions  across  stages  based  on  the  data 
we  had  available,  then  as  more  information  became  apparent  or  relevant,  we  spiraled  back  and  added  more 
detail  across  stages.  In  our  model,  as  is  Tessmer  and  Wedman's  (1990)  Layers  of  Necessity  Model,  "each 
layer  is  a  merged  set  of  tasks  or  questions  that  cut  across  the  discrete  stages  of  traditional  models.  Layers 
are  not  distinguished  by  the  type  of  task  perse,  but  by  the  level  of  complexity  associated  with  the  tasks  in 
that  layer."  (p.  8 1 ).  As  we  revisited  the  same  material,  at  different  times,  for  different  purposes,  we  built  on 
ideas  generated  at  previous  levels  in  iterative,  knowledge  building  cycles  (Rowland,  1992). 

While  [traditional]  models  may  have  iterative  features  that  allow  for  a  reconsideration  of 
earlier  design  activity  outputs,  they  emphasize  closure  of  each  component  in  the  process 
to  serve  as  input  tc  the  next  component.  A  layered  approach  assumes  that  components  of 
the  ID  process  will  .'^e  repeated  to  a  greater  degree  of  precision  and  sophistication  in 
subsequent  layers  of  the  process.  This  repetition  is  not  for  the  purpose  of  revision  earlier 
components  ...  but  of  adding  onto  the  work  that  was  done  earlier.  (Tessmer  &  Wedman, 
1990,  p.  80) 

Conclusion 

We  approached  the  design  process  with  the  belief  that  believe  that  instructional  design  requires  us 
to  make  decisions  based  on  our  assumptions  about  "how  people  think  and  learn  rather  than  mechanically  to 
apply  procedures  laid  out  in  an  instructional  design  model"  (Winn,  1991,  p.  38).  A  set  of  assumptions 
about  the  nature  of  the  learner  and  the  process  of  learning  led  to  implications  for  the  instructional  design 
process.  As  we  transformed  recommendations  for  constructivist  learning  environments  to  the  process  of 
designing  materials  from  a  constructivist  perspective,  we  attempted  to  embrace  the  complexity  of  the  design 
process,  recognizing  social  negotiations  as  an  integral  part  of  designing  the  materials.  Furthermore,  we 
emphasized  client-centered  design,  examining  the  instructional  content  from  multiple  perspectives  and 


nurturing  reflexivity  in  the  design  process.  As  we  analyzed  our  design  process,  we  found  that  it  differed 
from  traditional  models  of  instructional  design  in  three  primary  ways.  1 )  We  did  not  proceed  through  the 
steps  prescribed  by  traditional  instructional  design  models  (e.g.  Dick  and  Carey,  1990)  in  a  linear  fashion; 
however,  we  addressed  most  of  the  questions  posed  by  the  model.  2)  We  found  our  emphasis  shifted  from 
the  procedures  of  instructional  design  to  the  process  involved  in  making  the  decisions  that  guided  the 
development  process.  3)  Rather  than  following  a  step-by  step  progression  in  our  decision  making,  we 
addressed  the  questions  of  design  in  a  spiral  fashion,  progressing  through  a  series  of  steps  at  one  level,  then 
spiraling  back  and  adding  more  detail  within.  Just  as  constructivist  teachers  believe  that  "knowledge  is 
constructed  by  learners  as  they  attempt  to  make  sense  of  their  experiences"  (Driscoll,  1994,  p.  360),  these 
materials  were  constructed  by  the  designers  as  they  attempted  to  make  sense  of  the  information  available: 
Initially,  knowledge  of  the  learners'  characteristics,  needs  that  existed  in  science  methods  instruction,  the 
purpose  of  the  instruction,  and  the  environment  in  which  the  materials  would  be  used,  and  later,  the 
videotaped  footage  available. 

The  intent  of  this  discussion  was  not  to  prescribe  a  set  series  of  events  that  should  occur  in 
designing  materials  for  use  in  a  constructivist  learning  environment.  Rather,  we  wanted  to  share  our 
process  in  order  to  provide  one  experience  that  conveys  the  way  the  concept  of  "design"  fits  into  developing 
such  materials,  to  illustrate  that  decisions  are  not  random,  but  proceed  intentionally  and  purposefully, 
continually  adding  detail  to  prior  layers.  We  also  suggest  that  individuals  trained  in  traditional  instructional 
design  methodologies  can  work  in  concert  with  constructivists  to  create  materials  in  a  purposeful  manner. 
When  designing  materials  to  be  used  in  constructivist  learning  environments,  instructional  designers  should 
be  guided  by,  but  not  limited  by,  the  decisions  required  by  traditional  instructional  design  models.  With  a 
knowledge  of  the  questions  inherent  in  each  stage  of  traditional  instructional  design  models,  designers  can 
examine  the  data  that  evolve  through  the  construction  of  instructional  materials,  and  make  decisions  in 
collaboration  with  the  client  based  on  shared  assumptions  about  the  content  and  teaching/learning  process. 
We  hope  this  discussion  of  our  experiences  will  inform  instructional  designers  trained  in  traditional 
instructional  design  models  of  ways  to  adapt  their  training  to  the  altered  requirements  of  designing 
instruction  in  a  manner  consistent  with  constructivist  theory. 
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Information  Processing  Strategies  Used  in  Learning  from  Video 

Research  on  the  manner  in  which  learners  process  text  materials  has  allowed  the  development  of  a 
rather  comprehensive  theory  of  text  processing;  however,  there  is  less  information  available  on  how 
learners  process  information  presented  through  television  or  videotape  (1). 

Videotape  differs  from  text  in  several  ways.  Kozma  (1)  distinguished  among  various  type  of  media 
in  terms  of  their  symbol  systems,  processing  capabilities,  and  technology.  The  technology  of  text  and 
videotape  affect  learning  only  to  the  extent  that  access  to  information  present  on  videotape  is  limited  to 
locations  where  a  videotape  playback  machine  and  monitor  are  present,  whereas  access  to  text  is  much  less 
restricted  by  location.  However,  the  symbol  systems  and  processing  capabilities  of  the  two  media  may 
affect  the  nature  of  the  information  gained  and  the  way  the  information  is  processed.  Text  presents  verbal 
information  through  printed  words  and  uses  pictures,  diagrams  and  other  representational  symbol  systems  to 
present  visual  information.  With  videotape,  the  verbal  information  primarily  is  presented  orally.  Pictures 
and  other  representational  images  often  depict  motion  and  are  usually  presented  simultaneously  with  the 
verbal  information.  Text  allows  the  reader  to  access  the  information  at  random,  but  videotapes  present 
information  in  a  constant  stream  of  auditory  sounds  and  visual  motion.  Videotapes  present  information  in  a 
linear  format,  progressing  without  pause  between  the  beginning  and  end  of  the  program. 

The  technology,  symbol  systems,  and  processing  capabilities  of  a  medium  interact  to  provide 
unique  opportunities  for  information  access.  For  example,  I  may  be  able  to  view  a  videotape  demonstrating 
how  to  change  a  tire  on  a  car  in  my  home  prior  to  attempting  the  task.  The  videotaped  demonstration 
would  allow  me  to  see  the  specific  items  and  procedures  referred  to  by  the  verbal  information.  As  such,  I 
may  be  able  to  better  "understand"  the  specific  steps  involved  in  changing  a  tire.  I  can  seethe  results  of 
improper  actions  at  various  stages  in  the  process.  However,  due  to  the  nature  of  the  technology,  I  can  not 
take  the  videotape  with  me  to  the  location  where  I  want  to  change  a  tire.  With  text-based  information  on 
the  same  topic,  I  am  able  to  use  the  materials  as  a  reference  source  while  performing  the  task  of  changing 
my  tire.  The  symbol  system  are  limited  to  the  printed  word  and  still  pictures,  and  I  am  not  able  to  see  the 
steps  performed  by  another  person.  However,  due  to  the  random  access  provided  by  text,  I  am  able  to 
quickly  locate  references  to  specific  steps  where  I  may  need  additional  information. 

Role  of  Prior  Knowledge 

These  differences  in  the  processing  capabilities  of  text  and  video  may  be  especially  acute  for 
learners  with  prior  knowledge  of  the  content.  Perhaps  users  have  changed  tires  on  other  automobiles. 
They  may  be  familiar  with  the  steps  of  removing  the  hub  cap,  removing  the  lug  nuts,  and  so  forth,  but 
may  have  never  used  the  particular  type  of  jack  that  came  with  their  cars.  With  text-based  materials,  they 
can  quickly  skip  over  the  steps  on  which  they  need  no  more  information  and  go  rather  directly  to  the 
information  needed.  Videotape  does  not  provide  such  easy  random  access  to  specific  information. 

Given  the  differences  in  the  ease  with  which  learners  may  randomly  access  text  and  video-based 
information,  it  may  be  useful  to  look  at  how  these  capabilities  are  used  by  learners  reading  text.  Bazerman 
(2)  found  that  experts  in  the  domain  of  physics  searched  the  literature  for  new  information  to  update  their 
knowledge  in  a  random  manner,  driven  by  "purpose  laden  schemata".  The  physics  experts  skimmed  articles 
in  their  content  area  very  quickly  and  randomly.  Often  they  began  reading  the  abstract  or  conclusion,  and 
once  they  determined  that  the  article  was  of  personal  value,  they  often  skipped  to  certain  sections.  Their 
access  to  information  was  guided  by  their  search  for  very  specific  information  to  complement  their  existing 
knowledge.  They  appeared  to  be  seeking  answers  to  very  specific  questions,  seeking  information  that  fit 
within  their  existing  schemata  of  physics  and  the  way  research  should  be  done.  It  is  unlikely  that  novices 
would  have  been  able  to  read  articles  reporting  physics  research  in  a  similar  way.  Instead,  it  is  likely  that 
they  would  have  needed  to  start  at  the  beginning  and  carefully  construct  a  mental  model  of  the  content  in 
order  to  understand  the  message  of  the  article. 

It  is  obvious  that  text  can  be  explored  differently  based  on  the  prior  knowledge  that  users  bring  to 
the  reading  task.  Due  to  its  stability,  the  reader  has  "random  assess"  to  the  information  on      printed  page. 
Although  text  information  is  primarily  designed  to  be  read  in  a  linear  sequence,  the  stability  of  te:;t  allows 
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the  users  to  access  the  information  in  a  sequence  of  their  own  choosing.  Yet  even  when  reading  in  a  linear 
sequence,  the  learners'  prior  knowledge  may  affect  the  way  they  process  the  information 

Steinley  (3)  compared  the  processing  styles  of  readers  as  they  read  passages  on  word  games  with 
which  they  were  familiar  (crossword  puzzles)  and  unfamiliar  (doublets).  He  asked  them  to  compare 
crossword  puzzles  and  doublets  with  word  search  games  (with  which  they  were  familiar).  Readers  were 
instructed  to  pause  at  the  end  of  each  paragraph  and  note  whether  they  were  attempting  to  "comprehend"  the 
information  or  "compare  and  contrast"  the  information  with  word  search  games.  As  predicted,  readers  spent 
more  time  comparing  and  contrasting  crossword  puzzles  with  word  search  games  than  in  comparing  and 
contrasting  the  unfamiliar  game  of  doublets  with  word  search  games.  Conversely,  the  researchers  found  that 
readers  spent  significantly  more  of  their  time  trying  to  comprehend  passages  on  the  word  game  unfamiliar 
to  them  than  on  the  word  game  familiar  to  them.  As  readers  concentrated  on  comprehending  unfamiliar 
topics,  they  based  their  selection  of  a  comprehension  strategy  on  limited  background  knowledge,  the 
complexity  of  the  game,  or  their  lack  of  information  necessary  to  compare  or  contrast  the  game  with  a  more 
familiar  game.  When  reading  about  the  familiar  topic  of  crossword  puzzles,  readers  also  indicated  the  use 
of  a  comprehension  strategy  fairly  often,  but  provided  much  different  reasons  for  doing  so.  When  reading 
text  on  the  familiar  topic,  they  selected  a  comprehension  strategy  to  attend  to  details,  to  orient  themselves 
to  the  text  passage,  and  when  nothing  in  the  paragraph  was  useful  for  comparisons. 

Consistent  with  mental  model  theory,  these  findings  suggests  that  learners  build  a  mental  model  of 
new  content  from  the  bottom-up,  while  they  process  information  on  a  familiar  topic  in  a  top-down  manner, 
calling  upon  existing  knowledge  and  modifying  it  as  they  attend  to  relevant  new  information  (4).  Whereas 
novices'  knowledge  is  limited  and  organized  around  the  explicit  information  presented,  experts  reorganize 
new  information  around  principles  and  abstractions  of  their  prior  knowledge  on  the  subject  (5).  The  studies 
cited  above  suggest  that  learners  with  prior  knowledge  of  a  text  topic  may  choose  to  access  the  information 
randomly  or  engage  in  actively  comparing  new  information  with  their  prior  knowledge.  However,  due  to 
the  transient  nature  of  information  presented  through  videotape  and  the  linear  nature  of  the  medium,  learners 
may  be  unable  to  process  video-based  instruction  in  the  same  manner  as  they  process  text-based  instruction. 

Purpose  of  the  Study 

The  purpose  of  this  exploratory  study  was  to  examine  the  way  that  learners  process  videotaped 
materials  on  a  familiar  or  unfamiliar  topic  as  informed  by  text-based  research.  A  variation  of  the  think- 
aloud  method  of  assessing  cognitive  processes  was  used  to  gain  insight  into  the  role  of  prior  knowledge  in 
learners'  thoughts  while  viewing  a  videotape  on  the  topic  of  instructional  design.  Sasse  (6)  suggested  that  a 
teach-back  technique,  where  learners  are  asked  to  "teach"  a  subject  to  a  new  learner,  may  be  an  effective  way 
to  determine  learners'  mental  models  of  a  content  area;  thus,  the  teach-back  method,  in  combination  with 
think-aloud  protocols,  was  used  to  assess  the  manner  in  which  the  content  was  incorporated  with  the 
learners'  prior  knowledge.  It  was  expected  that  learners  with  prior  knowledge  of  instructional  design  would 
make  more  comments  that  compared  and  contrasted  the  new  information  with  their  prior  knowledge  than 
those  unfamiliar  with  the  topic  of  the  videotape. 

Methods 

Participants 

The  nine  participants  in  this  study  were  enrolled  in  a  graduate  class  in  video  production  at  a  large 
midwestern  university.  Participants  were  classified  as  experienced  in  instructional  design  or  unfamiliar 
with  the  topic  based  on  prior  exposure  to  the  topic  of  instructional  design  in  coursework  or  practice. 
Students  were  asked  whether  they  a)  had  ever  taken  a  course  in  instructional  design,  b)  had  designed 
instructional  materials,  and  c)  were  familiar  with  any  instructional  design  models.  Five  students  responded 
positively  to  at  least  one  of  these  three  questions,  and  four  responded  negatively.  One  student  experienced 
in  instructional  design  was  dropped  from  the  analysis  due  to  technical  problems  in  recording  her  responses, 
thus,  the  final  sample  consisted  of  four  experienced  participants  and  four  participants  unfamiliar  with  the 
topic. 

Matet  :ais 

The  videotape  used  in  this  study  described  the  development  of  a  course  using  a  model  of 
instructional  design  unique  to  a  particular  organization.  Although  the  steps  in  the  design  process  are 
similar  to  those  in  the  model(s)  with  which  some  of  the  participants  were  familiar,  the  steps  were  clustered 
differently  and  the  tape  used  different  terminology.  None  of  the  participants  were  familiar  with  the 
particular  model  of  design  presented  in  the  videotape.  The  videotape  provided  a  general  overview  of  the 
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components  of  the  instructional  design  process  and  the  people  involved.  The  program  then  presented  the 
details  of  each  of  the  five  stages  in  the  instructional  design  model  in  sequence.  The  tape  described  the  steps 
involved  in  each  stage  of  design  and  the  roles  of  the  individuals  involved  in  completing  the  steps.  For  the 
purposes  of  this  research,  the  videotape  was  divided  into  six  segments  ranging  in  length  from  1.5  to  4 
minutes. 

Data  Collection 

Think-aloud  protocols  have  been  used  to  examine  the  cognitive  processing  of  experts  and  novices 
in  problem  solving  tasks  (7)  and  in  research  on  learners'  mental  models  (6).  Thus  we  believed  that  the 
think-aloud  method  would  allow  us  to  determine  the  extent  to  which  viewers  of  a  videotape  on  a  familiar  or 
unfamiliar  topic  were  attempting  to  comprehend  new  information  in  order  to  construct  a  mental  model  of 
the  topic  or  compare  and  contrast  the  new  information  with  their  prior  mental  models.  However,  when 
processing  text  materials,  readers  are  able  to  pause  in  their  reading  to  orally  express  their  ideas.  Due  to  the 
continual  pace  of  auditory  and  visual  information  presented  via  videotape,  verbalizations  during  viewing  are 
likely  to  interfere  with  attending  to  information  at  the  pace  at  which  it  is  presented.  Although  viewers 
could  pause  the  videotape  to  verbalize  their  ideas,  we  felt  the  freedom  to  pause  the  videotape  to  "think- 
aloud"  about  it's  content  would  provide  viewer  control  of  the  pace  of  information  presentation  and  represent 
an  artificial  manipulation  of  the  typical  videotaped  or  televised  presentation.  Thus,  in  this  study,  the 
viewers  were  asked  to  "think-aloud"  at  the  end  of  each  of  the  six  subsections  in  the  videotape. 

At  the  end  of  each  subsection  (ranging  in  length  from  1 .5  to  4  minutes),  the  interviewer  stopped 
the  videotape  and  asked  the  participant  to  verbalize  his  or  her  thoughts  during  the  previous  section.  The 
first  section,  which  provided  a  general  introduction  to  the  instructional  design  model  and  personnel,  was 
used  for  the  participants  to  practice  the  think-aloud  method.  The  think-aloud  data  consists  of  the 
verbalizations  which  followed  the  five  remaining  sections,  corresponding  to  the  five  stages  in  the  design 
model.  The  verbal  data  were  audiotaped,  coded  by  participant  number,  and  transcribed  for  analysis. 

A  teach-back  technique  was  used  to  assess  the  learners'  understanding  of  the  tape's  content.  At  the 
end  of  the  videotape,  the  participants  were  asked  to  "teach"  the  instructional  design  process,  as  presented  in 
the  videotape,  to  the  interviewer.  The  verbal  responses  to  the  teach-back  task  were  audiotaped,  coded  by 
participant  number,  and  transcribed  for  analysis. 

Procedures 

Each  participant  met  with  the  interviewer  individually  in  a  quiet  room.  All  interactions  were 
audiotaped.  When  the  participant  arrived,  he  or  she  was  greeted  by  the  interviewer,  questioned  on  prior 
experience  with  instructional  design  and  with  this  particular  design  model,  and  informed  of  the  topic  of  the 
videotape  and  the  task  for  the  experimental  session.  They  were  told  that,  at  the  end  of  the  program,  they 
would  be  asked  to  teach  the  method  of  instructional  design  described  in  the  videotape  to  the  interviewer  and 
informed  that  they  could  take  written  notes  if  desired.  Participants  were  instructed  in  the  think-aloud 
method.  They  were  told  that  the  interviewer  would  stop  the  videotape  from  time  to  time,  and,  at  that  time, 
they  were  to  orally  recall  their  thoughts  during  the  previous  section  of  the  tape. 

As  the  participants  watched  the  videotape,  the  interviewer  stopped  the  videotape  at  the  end  of  each 
of  the  six  sections  and  the  participants  orally  reflected  back  on  their  thoughts  during  the  previous  section. 
The  first  section  of  the  videotape  was  used  to  "practice"  the  think-back  method,  thus,  no  data  was  collected 
from  this  section.  At  the  end  of  the  videotape,  the  viewer  was  asked  to  describe  any  thoughts  he  or  she  had 
on  the  program  as  a  whole,  then  asked  to  describe  the  model  presented  in  the  videotape  as  if  he  or  she  was 
teaching  it  to  a  classmate. 

Data  Analysis,  Results,  and  Discussion 

The  think-aloud  data  and  teach-back  data  were  analyzed  to  determine  differences  between  those 
familiar  and  unfamiliar  with  the  topic.  It  should  be  noted  that  this  was  an  exploratory  study,  thus  no 
hypotheses  were  formally  tested.  Initially,  overall  frequencies  and  percentage  scores  of  coded  categories  were 
tabulated  for  the  think-aloud  and  teach-back  data.  Finally,  the  think-aloud  and  teach-back  protocols  were 
analyzed  qualitatively  to  complement  and  inform  the  patterns  discerned  in  the  quantitative  data. 

Think-alouds 

The  coding  of  the  think-alouds  protocols  followed  the  recommendations  of  Ericsson  and  Simon 
(7).  The  verbatim  transcriptions  of  the  think-aloud  data  were  divided  into  statements  representing  complete 
thoughts.  The  statements  from  each  participant  were  coded  as  to  participant  number  then  jumbled  so  that 


46  ^0 


the  interpretation  of  one  idea  was  not  influenced  by  the  participant's  prior  statements.  In  order  to  develop 
coding  categories,  one  researcher  read  the  entire  list  of  statements  and  looked  for  themes  to  emerge  from  the 
participants'  verbalizations.  She  identified  a  set  of  seven  coding  categories.  Statements  seemed  to  refer  to 
1 )  the  viewers'  comprehension  of  th;  material,  2)  positive  evaluations  of  their  comprehension,  3)  negative 
evaluations  of  their  comprehension,  4)  the  instructional  design  process,  5)  the  quality  of  the  video,  6) 
distractive  aspects  of  the  video,  and  7)  other  miscellaneous  comments.  Using  this  set  of  coding  categories, 
two  other  researchers  independently  coded  the  statements  into  these  categories.  Both  researchers  then  sorted 
the  statements  by  coding  category  and  examined  the  lists  to  determine  if  all  of  the  statements  coded  into  a 
particular  category  were  of  a  similar  nature.  Following  this  self-check  procedure,  the  two  researchers 
compared  their  coding  of  each  statement.  Differences  in  coding  were  discussed  until  the  researchers  came  to 
agreement.  The  majority  of  coding  differences  occurred  due  to  the  difficulty  that  the  coders  had  in 
discriminating  between  statements  belonging  in  the  two  categories  of  "distracted  by  visuals"  and  "quality  of 
video".  Due  to  difficulties  in  definitively  placing  certain  statements  into  one  of  the  two  categories 
("distracted  by  visuals"  and  "quality  of  video") ,  they  were  collapsed  into  one  ("quality  of  video"). 
Definitions  of  the  coding  categories  and  example  statements  are  presented  in  Figure  1 : 
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PnHina  CatpaoripQ  for  Think -aloud  Protocol 

Comprehension 

statements  reflect  an  attempt  to  comprehend  the  content  of  the 
program: 

"  They  went  over  objectives,  content,  topic  activities,  scripts, 
budget,  and  then  the  design" 

Positive  evaluations 
of  comprehension 

statements  are  positive  affirmations  of  one's  comprehension: 
"It  makes  pretty  good  sense. " 

Negative  evaluations 
of  comprehension 

statements  are  negative  comments  on  one's  comprehension: 
"It's  confusing. " 

Design 

statements  compare  or  contrast  the  information  with  prior 
knowledge  of  instructional  design: 

"They  follow  the  instructional  design  process  that  I'm  familiar  with. " 

Oualitv  of  video 

statements  comment  on  the  quality  of  the  visuals  or  audio: 
"I  thought  they  could  have  been  a  little  more  creative". 
Some  statements  indicate  the  viewer  was  distracted  by  the 
visuals  or  audio: 

7  was  spending  a  lot  of  time  looking  at  where  people  were  in  their 
office  and  where  they  were  going  next. " 

Other 

statements  compare  or  contrast  the  information  with  other  prior 
knowledge: 

"  Part  of  it  I  related  to  a  class  I'm  taking  right  now  where  we're 
talking  about  reliability  and  validity." 

Following  the  coding  of  each  statement,  the  percentage  of  statements  falling  in  each  category  was 
calculated  by  group  (experienced  in  instructional  design  and  unfamiliar  with  the  topic).  As  you  will  notice 
from  Table  1,  the  mean  percent  of  comments  were  very  similar  between  the  two  groups  in  a  number  of 
coding  categories.  Participants  who  were  experienced  in  instructional  design  (E)  and  those  unfamiliar  with 
the  topic  (U)  had  a  similar  percent  of  comments  relating  new  information  to  their  prior  knowledge  of 
design,  relating  new  information  to  other  prior  knowledge,  and  negative  evaluations  of  their 
comprehension.  Those  experienced  in  instructional  design  had  a  greater  percentage  of  comments  that 
positively  evaluated  their  comprehension.  There  were  greater  differences  in  the  percent  of  comments  made 
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in  two  other  categories:  Participants  unfamiliar  with  the  content  made  a  greater  percent  of  comments  related 
to  the  quality  of  the  video  than  those  experienced  in  instructional  design;  however,  experienced  participants 
made  a  greater  percent  of  comments  indicating  comprehension  of  the  content  than  those  who  were 
unfamiliar  with  the  topic  of  the  videotape. 


Table  1:    Think-aloud  task:    between  group  percentages  and  standard  deviations 


Coded  category 

QV 

C 

D 

N 

P 

0 

Experienced 
X% 

22.1 

41 

15 

13 

7 

2 

(SD) 
Unfamiliar 

(2.6) 

(11) 

(3) 

(3) 

(1) 

(1) 

X% 

53.4 

15 

14 

12 

1 

4 

(SD) 

(5.8) 

(0.5) 

(3) 

(3) 

(1) 

(1) 

Note:     QV:  Quality  of  video  N:      Negative  evaluations  of  comprehension 

C:      Comprehension  P  :      Positive  evaluations  of  comprehension 

D:      Design  process  O:      Other  statements 


The  similarities  between  the  two  groups  in  the  percentage  of  comments  comparing  or  contrasting 
the  information  in  the  tape  with  prior  knowledge  of  design  was  surprising  and  prompted  a  review  of  the 
individual  think-aloud  protocols.  Although  the  participants  classified  as  unfamiliar  with  the  topic  of 
instructional  design  indicated  that  they  had  never  had  a  course  in  the  topic,  had  never  designed  instructional 
materials,  and  were  not  familiar  with  any  instructional  design  models,  their  comments  during  the  think- 
aloud  protocols  indicated  that  they  were  interpreting  the  content  of  the  videotape  relative  to  knowledge 
gained  through  other  courses  and  work  experience.  The  following  comments  from  the  protocols  of  those 
unfamiliar  with  the  process  of  instructional  design  illustrate  how  they  were  attempting  to  relate  the  content 
of  the  videotape  to  their  prior  knowledge: 

It's  sort  of  the  pattern  we  have  learned  in  EDCl  class,  talking  about  the  same  things...  whether  it's 
feasible  or  not,  the  objectives,  the  outline,  and  that  kind  of  stuff.  So  it's  just  like  all  those  things  that 
I  already  learned  in  that  class. 

Like  I  said  before,  it's  like  may  EDCl  class.  It's  pretty  much  the  same  things  that  we're  doing  in  our 
project.  /   11  realized  that  I  was  thinking  that  right  now  we're  about;to  do  the  same  things  they're 
talking  bout  in  our  class  projects. 

Well,  again,  that's  getting  to  some  material  I  already  covered,  but  what  I  was  thinking  about  was  that 
I'm  doing  a  project  for  one  of  my  courses  right  now,  developing  instruction  for  a  problem  I  have  a  t 
work  and  I  was  thinking  when  they  were  giving  different  drafts  and  so  forth  that  this  material...  I 
guess....  and  I  guess  I  was  thinking  a  lot  of  time  looking  at  where  people  were  in  their  office  and  where 
they  were  going  next...  and  again,  that  could  be  because  I've  had  this  material  before... 

From  these  comments,  it  appears  that  the  participants  who  indicated  that  they  were  unfamiliar  with 
instructional  design  began  to  sec  the  similarity  between  information  they  had  been  exposed  to  in  other 
classes  and  the  process  of  instructional  design.  It  must  be  noted  that  all  of  the  participants  were  involved 
in  the  pre-production  planning  stages  of  videotape  production.  Although  they  may  have  been  unfamiliar 
with  the  term  "instructional  design",  they  began  to  see  that  they  were  familiar  with  some  aspects  of  the 
process. 

Within  group  variation,  as  indicated  by  the  wide  standard  deviations  in  some  categories,  also 
prompted  a  closer  examination  of  individual  protocols,  (see  Figure  2). 
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figure  2:    Think  aloud  frequencies  bv  grqup 
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Participants  unfamiliar  with  topic  of  instructional  design 


Note:  QV:  Quality  of  video 
C:  Comprehension 
D:      Design  process 


N:  Negative  evaluations  of  comprehension 
P:  Positive  evaluations  of  comprehension 
O:      Other  statements 


One  designer  had  a  large  number  of  additional  comments  related  to  comprehension  of  the  program.  When 
etmSg  his Comments  coded  as  "comprehension"  in  the  context  of  adjacent  comments,  the  vast  majonty 
of  them  were  related  to  trying  to  understand  how  he  would  teach  the  material: 

/  was  twine  to  get  it  down  so  I  could  reteach  it.  I  was  trying  to  get  as  much  of  it  down  as  I  could.  / 
uJers?Zit  I  understand  it  from  what  I've  done,  so  1  was  mainly  just  focusing  on  getttng  the  mam 
points  down. 

I  was  still  trying  to  get  all  the  parts  down.  All  the  different  pieces  of  the  puzzle,  because  it  makes 
sense.  It  fits  with  what  I  already  know. 

Generally,  I  was  just  trying  to  get  it  down  and  think  about  how  I  would  present  it. 
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As  this  individual  was  a  teacher  and  was  told  that  he  would  be  expected  to  teach  the  instructional  design 
model  to  the  interviewer  after  viewing  the  tape,  it  was  not  surprising  that  he  was  co^cenSg  on 
comprehending  the  informaf-on  in  ways  that  would  facilitate  the  teaching  of  the  content 

a-  „  li re? ad'ng  the  endre  Protocols  for  ^ch  individual,  two  of  the  experienced  designers  seemed  verv 
distracted  by  elements  related  to  the  design  of  the  video:  pwiencea  Designers  seemed  very 

There's  a  lot  of  audio.,  just  with  all  he's  talking  about...  and  he's  going  from  scent  to  scene... 
Between  those  particular  segments,  I  guess  there  was  a  lot  of  music.  I  tried  to  tune  it  out.... " 

lideosh°->        1  ^  mnkin8  WS  Vide°  C°Uld  ^  SeVm  °r  eight  dWerent  Vide°S-  different  me-hour 

ZtVaToTsZThfnZ  ^  Chalkbmrd-  *  ^  driVb}8  ^  '  th°U8ht  ^  Shmld  ^  ^ 

nn  th     Arl  °f        U"familia^ with  the  content  of  'he  videotape  focused  the  majority  of  their  comments 
on  the  quality  of  the  videotape  (two  individuals  in  particular),  as  indicated  in  the  following  comment 

I  actually  lost  track  of  him  at  onetime..  I  was  looking  at  the  light  on  the  bald  guy's  head,  thinking 
that  they  should  have  done  the  lighting  different.. " 

I  actually  didn  >t  hear  the  first  thing  he  was  saying  because  I  was  thinking  about  the  previous  section  if 

those  people  were  actors  or  the  actual  people  like  the  writer,  etc.  and  that's  about  all  I  was  thinking 
about  in  that  segment.  6 

I  felt  that  there  would  have  been  a  better  way  to  do  Ms,  like  with  graphics  and  bullets  or  something 
of  course,  this  could  be  because  of  the  course  I'm  taking  in  my  video  class...  I  think  that  putting  things 
on  the  board  took  too  much  time,  and  I  found  myself  watching  him  spell  instead  of  listening... 

It  appears  that  all  participants  were  attempting  to  make  sense  of  the  content  of  the  videotape 
relative  to  their  pnor  knowledge.  Those  experienced  in  instructional  design  had  the  background  necessary  to 
comprehend  the  content  of  the  tape.  However,  rather  than  focusing  on  the  process  of  instructional^. 

tt  It^l     S  th°Se„Unuamiliar  With  thC  pr°CeSS  concentr^d  on  evaluating  the  production  values  of 
the  videotape.  Although  all  the  Part.c,pants  were  enrolled  in  a  course  on  videotape  production  and  each 
participant  possessed  a  similar  awareness  of  good  production  techniques,  those  unfamiliar  with  the  topic  of 
instructional  design  may  have  been  unable  to  build  a  coherent  mental  model  of  instructional  design;  thus 
SnTques      "         *°  vide0tape  based  on  their  mental  ™dels  of  good  video  production 

Teach-backs 

Using  a  procedure  described  by  Pask  and  Scott  (8),  the  audiotaped  teach-back  protocols  were 
transcnbed  and  ,ie  statements  were  coded  into  categories  used  by  Pask  and  Scott.  Unlike  the  analysis  of 
the  think-aloud  protocols  which  were  coded  as  independent  thoughts,  the  coding  of  the  teach-back 
statements  required  an  awareness  of  ideas  that  came  before  and  after  the  statement  to  be  coded 

Initially,  statements  were  coded  using  Pask  and  Scott's  categories  (8);  however,  the  need  to  modify 
the  categories  to  encompass  the  data  set  soon  became  apparent.  The  final  coding  categories  were  determined 
though  a  revlew  of  the  teach-back  protocols  to  determine  categories  of  responses.  Two  researchers,  unaware 
of  the  group  to  which  the  participants  belonged,  independently  read  the  protocols  to  determine  trends  in  the 
data.  Both  researchers  identified  similar  trends  in  the  data,  and  coding  categories  were  finalized  This 
analysis  resulted  in  the  elimination  of  certain  categories  used  by  Pask  and  Scott  (8)  and  the  subdividing  of 
others.  Both  researchers  then  coded  the  teach-back  protocols  independently.  Then,  the  two  researchers 
compared  their  coding  of  each  statement.  Differences  in  coding  were  discussed  until  the  researchers  came  to 
agreement.  Definitions  of  the  coding  categories  used  in  the  final  analysis  and  example  statements  are 
presented  in  Figure  3: 


Fieure  3:    Codine  Scheme  for  Teachbacks 

Advance  organizers 

preview  information  to  come: 

"I  can  tell  you  of  the  overall  design  process  from  [lists  steps]". 

Reviews  or  summaries 

review  or  summarize  previously  presented  information: 

"Once  you 've  established  your  objectives.. " 

People 

mentioned  people  from  the  videotape: 

"there  is  the  director  for  the  program  part  of  it.. " 

Steps 

mention  steps  in  the  instructional  design  process 
"you're  assessing  your  training  needs. " 

Other 

recall  other  information  directly  from  the  tape 

(other  than  steps  in  the  process  or  the  people  involved): 

"...  costs  quite  a  bit  of  money  and  time". 

Elaborations 

provide  additional  information  on  the  people,  steps,  or  other 

information  derived  from  the  tape: 

"...they're  the  ones  who  actually  do  the  testing" 

Inferences 

derived  from  information  stated  in  the  tape  or  from  prior 
knowledge: 

"/  call  that  the  subject  matter  expert". 

Table  2:    Tcachback  task:    between  group  percentages  and  standard  deviations 


Coded  category 

A 

P 

S 

E 

I 

R 

O 

Experienced 

X% 

6.2 

15 

45 

1 1 

16 

4 

2.5 

(SD) 

(2.6) 

(4.4) 

(9) 

(5) 

(5) 

(2) 

(.08) 

Unfamiliar 
X% 

5.5 

15 

43 

23 

7 

4 

1.6 

(SD) 

(2.1) 

(3.8) 

(6) 

(8) 

(1) 

(2) 

(1) 

Note:     A:  Advance  organizers  P:  People  mentioned  in  tape 

S:  Step  in  design  process  E:  Elaborations 

I:  Inferences  R'-  Review  or  summary  statements 

O:  Other  information  recalled 


Following  the  coding  of  each  statement,  the  percentage  of  statement  falling  in  each  category  was 
calculated  by  group.  As  you  will  notice  from  Table  2,  there  was  little  difference  in  the  percentage  of 
comments  coded  as  advance  organizers ,  people  mentioned ,  steps  mentioned,  review  or  summary 
statements,  or  other  information  recalled  directly  from  the  tape.  However,  participants  who  were  unfamiliar 
with  instructional  design  elaborated  on  the  information  recalled  from  the  tape  to  a  greater  extent  than  those 
experienced  in  the  process  .  Those  experienced  in  instructional  design  expressed  more  inferences  derived 
from  the  tape  or  prior  knowledge  than  those  unfamiliar  with  the  process. 

As  indicated  in  Figure  4,  there  was  less  within  group  variation  in  the  teach-back  data.  One 
individual  in  the  group  of  experienced  designers  had  fewer  comments  coded  as  "Steps"  than  the  other  three, 
and  one  participant  unfamiliar  with  instructional  design  had  more  comments  coded  as  "Steps"  and 
"Elaborations"  than  the  other  three  inexperienced  participants. 
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Figure  4:    Teach-back  frequencies  by  group 
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Participants  unfamiliar  with  the  topic  of  instructional  design 

Nott:    A:  Advance  organizers  P:  People  mentioned  in  tape 

S:  Step  in  design  process  E:  Elaborations 

I:  Inferences  R:  Review  or  summary  statements 

O:  Other  information  recalled 


In  general,  both  groups  mentioned  a  similar  number  of  instructional  design  steps,  people  involved 
in  the  process,  and  other  information  stated  in  the  tape.  It  must  be  noted  that  participants  were  encouraged 
to  take  notes  to  aid  them  in  performing  the  teach-back  task  in  order  to  replicate  a  natural  setting  in  which  an 
instructional  videotape  would  be  used.  It  is  likely  that  the  participants  were  guided  by  their  notes  when 
teaching  the  process  back  to  the  interviewer,  thus,  both  groups  recalled  a  similar  number  of  steps,  people, 
and  other  details  from  the  tape.   Both  groups  organized  their  teach-backs  similarly,  with  both  groups  using 
a  similar  number  of  advanced  organizers  and  review  and  summary  statements. 

However,  when  asked  to  teach  the  instructional  design  method  to  another  person,  there  were 
differences  in  the  way  the  two  groups  approached  the  task:  those  with  experience  in  instructional  design 
included  a  greater  number  of  inferences  derived  from  the  tape  or  prior  knowledge,  while  those  unfamiliar 
with  the  process  included  a  greater  number  of  elaborations  on  the  information  recalled  from  the  tape. 
Inferences  reflected  information  nCLexplicitly  stated  in  the  tape,  but  which  could  be  derived  from  prior 
knowledge  or  by  linking  information  in  the  tape  in  a  new  way.  In  the  following  examples,  the  segments 
coded  as  inferences  could  not  have  been  derived  from  the  content  of  the  tape: 
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They  call  it  the  content  person  (coded  as  People) .  /  call  that  the  subject  matter  expert  (coded  as 
Inference) 


....you  actually  do  the  production  (coded  as  a  Step) ...  This  cost  quite  a  bit  of  money  here,  too.  (coded 
as  Inference). 

The  first  area  is  to  determine  the  feasibility  ( coded  as  a  Step)....  /  would  imagine  that's  when  you  go 
into  a  needs  analysis,  (coded  as  an  Inference) 

On  the  other  hand,  those  unfamiliar  with  the  process  of  instructional  design  included  a  greater 
number  of  elaborations  in  their  teach-backs.  Elaborations  differed  from  inferences  in  that  statements  coded 
as  elaborations  expanded  upon  the  information  recalled  from  the  tape.  They  offered  additional  details 
supporting  the  recall  of  steps,  people,  or  other  information  presented  in  the  tape.  Consider  the  following 
segment  from  the  teach-back  protocols  of  inexperienced  participants: 

You're  assessing  your  training  needs-  (coded  as  a  Step)...  does  it  need  to  be  done?  I  s  the  instructional 
design  and  development  the  right  things  to  do  based  on  the  subject  matter  expert,  background 
information,  (coded  as  an  Elaboration) 

Then  it  goes  into  development  (coded  as  a  Step)  where  they  go  ahead  and  piece  together  everything,  like 
it  needs  slides,  video,  music,  anything  like  that  they  are  going  to  need  (coded  as  Elaboration). 

Then  there's  the  design  report  (coded  as  a  Step)  so  that  it  actually  includes  the  blueprint  and  just 
summarizes  all  these  little  aspects  (coded  as  Elaboration). 

The  analysis  of  the  teach-back  protocols  suggests  that  experienced  participants  may  have  been  able 
to  built  from  prior  experience  to  generate  inferences  during  the  teach-back  sessions;  however,  participants 
unfamiliar  with  the  topic  of  instructional  design  may  have  used  the  opportunity  to  teach-back  the  topic 
presented  in  the  videotape  to  further  process  the  information  as  they  explained  it  to  the  interviewer  through 
elaborating  on  the  topics  as  they  spoke. 

Conclusion 

Based  on  the  research  conducted  with  text,  we  expected  that  those  familiar  with  the  topic  of  the 
videotape  would  compare  and  contrast  the  new  information  with  their  prior  knowledge  as  they  viewed  the 
videotape.  We  expected  them  to  seek  answers  to  questions  that  they  had  about  the  process  of  instructional 
design  specific  to  this  particular  organization  in  order  to  modify  their  existing  mental  models  of  the  topic. 
Conversely,  we  expected  those  unfamiliar  with  the  topic  to  concentrate  on  comprehending  the  information 
in  order  to  build  a  mental  model  of  the  content. 

The  data  from  the  think-aloud  protocols  suggest  that  all  participants  were  attempting  to  relate  the 
new  information  presented  via  videotape  to  topics  with  which  they  were  familiar.  While  those  with 
experience  in  instructional  design  did  compare  the  new  information  to  their  prior  knowledge  of  the  topic, 
those  unfamiliar  with  the  topic  appeared  to  be  relating  the  steps  in  the  instructional  design  process  to  the 
steps  in  planning  for  a  videotape  production.  Comments  such  as  "this  is  like  what  we're  doing  in  my 
education  class  "  were  common. 

In  addition,  all  of  those  unfamiliar  with  the  topic  and  half  of  those  with  experience  in  instructional 
design  were  frequently  evaluating  the  production  of  the  videotape  while  viewing  the  program.  As  an 
exploratory  study,  this  investigation  was  limited  by  the  fact  that  all  participants  were  enrolled  in  a  course 
which  offered  them  the  knowledge  needed  to  critically  examine  the  production  of  the  videotape.  However, 
these  statements  provide  further  evidence  that  the  viewers  relied  on  knowledge  gained  in  other  contexts  to 
interpret  what  they  saw  in  the  videotape. 

Analysis  of  the  teach-back  protocols  provided  additional  information  on  the  cognitive  processing  of 
the  videotaped  information.  Participants  with  experience  in  instructional  design  included  a  greater 
percentage  of  inferences  which  linked  the  content  of  the  videotape  with  other  related  ideas;  participants 
unfamiliar  with  the  topic  included  a  greater  percentage  of  comments  that  elaborated  on  the  content  of  the 
tape.  The  nature  of  the  comments  coded  as  inferences  suggests  that  the  experienced  participants  may  have 
stored  the  new  information  with  their  prior  knowledge  of  instructional  design,  and  thus,  as  they  recalled  the 
information  from  the  tape,  they  also  recalled  other  related  ideas.  The  nature  of  the  comments  coded  as 
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elaborations  suggests  that  those  unfamiliar  with  the  topic  of  the  videotape  may  have  continued  to  process 
the  content  of  the  tape  during  the  teach-back  sessions,  mentally  organizing  the  information  as  they 
elaborated  on  the  content  and  added  details  to  the  steps  recalled  from  the  tape. 

As  an  exploratory  study,  the  results  of  this  investigation  provide  evidence  that  the  think-aloud  and 
teach-back  methods  can  be  used  successfully  to  gain  insight  into  the  cognitive  processing  of  videotaped 
materials.  Prior  to  this  study,  no  published  studies  reported  using  the  think-aloud  or  teach-back  method  to 
collect  data  on  viewers'  cognitive  processing.  The  think-aloud  protocols  seemed  to  reasonably  represent  the 
viewers'  thoughts  during  the  program.  Future  studies  using  similar  procedures  shoujd  collect  student  notes 
to  complement  the  oral  data.  In  addition,  the  teach-back  method  seemed  to  provide  further  information  on 
the  learners'  cognitive  processing  of  the  information  presented  via  videotape.  Future  research  should 
continue  to  examine  the  relationship  between  learners'  prior  knowledge  of  the  content  of  a  videotaped 
presentation  and  the  way  they  process  the  information  presented  though  this  linear,  transient  medium. 
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General  Method 


This  study  analyzed  the  copyright  knowledge  levels  of  media  directors  of  selected  higher  education 
institutions.  The  study  determined  their  overall  knowledge  level  of  the  1976  Copyright  Act  by  questioning 
the  media  directors  with  respect  to  specific  areas  of  the  law  that  are  pertinent  to  their  responsibilities.  These 
specific  areas  included  Sections  106  (Exclusive  Rights),  107  (Limitations  on  Exclusive  Rights:  Fair  Use), 
110  (Limitations  on  Exclusive  Rights:  Exemption  of  Certain  Public  Performances),  and  the  guidelines  that 
cover  classroom  materials  taken  from  books  and  periodicals,  music,  and  off-air  videotaping.  Information  on 
demographics  was  also  collected.  In  addition,  follow-up  telephone  interviews  were  conducted  with  a  random 
sample  of  10%  of  the  respondents. 

Research  Population 

The  population  consisted  of  members  of  the  Division  of  Educational  Media  Management  in  the 
Association  for  Educational  Communications  and  Technology  (AECT)  who  are  employed  at  institutions  of 
higher  education  and  other  members  of  AECT  who  indicated  in  the  directory  that  they  have  some  media 
responsibilities  at  higher  educational  institutions.  The  1993  AECT  Membership  Directory  was  used  to 
identify  members  of  the  population.  Several  criteria  were  used  to  determine  if  a  subject  was  a  potential 
member  of  the  population.  These  included  the  subject's  position  title.  Members  of  the  sample  group  who 
were  chosen  had  titles  of  media  director,  media  specialist,  head  of  audio-visual,  director  of  learning 
resources,  or  similar  variations,  at  higher  educational  institutions.  The  AECT  directory  provides  two  other 
alternatives  to  locating  population  members.  The  first  is  the  division  affiliation  of  each  member.  The 
Division  of  Educational  Media  Management  (DEMM)  primarily  contains  educational  media  directors. 
However,  some  members  of  DEMM  are  involved  in  K-12  management.  Subjects  who  indicated  that  they 
were  members  of  DEMM  and  worked  at  higher  educational  institutions  were  considered  to  be  members  of 
the  sample  group.  Finally,  many  AECT  members  list  professional  descriptions  with  their  address  (i.e. 
Teacher/Professor,  Librarian,  Instrucuonal  Designer,  etc.)  Those  individuals  who  indicated  that  they  were  an 
AV  director  or  media  specialist  at  a  higher  educational  institution  were  also  included.  A  total  of  466 
subjects,  nationwide,  were  identified  as  being  probable  members  of  the  population  due  to  their  position 
titles  or  their  affiliations  within  the  organization.  Even  with  the  detailed  and  individual  selection  of 
population  members,  it  was  expected  that  a  minor  percentage  of  those  identified  may  be  eliminated  later, 
because  they  are  not  media  professionals  at  a  higher  educational  institution.  This  anticipated  reduction  in  the 
number  of  members  of  the  population  was  taken  into  account  when  determining  the  sample  size. 

The  cover  letter  included  with  the  mailed  instrument  identified  the  population  and  asked 
participants  to  complete  the  survey  only  if  they  were  members  of  the  population.  Demographic  questions 
on  the  survey  were  alsoused  to  screen  respondents  who  were  not  part  of  the  population. 

Specific  Procedures 

Each  of  the  466  members  of  the  sample  population  were  mailed  the  questionnaire.  Members  of 
the  sample  group  were  mailed  a  questionnaire  (multiple  choice  test)  that  asked  specific  questions  concerning 
the  Copyright  Law  of  1976.  In  addition,  a  cover-letter  describing  the  study,  and  a  self  addressed  stamped 
envelope  accompanied  the  questionnaire.  The  return  envelopes  were  coded  to  determine  who  had  responded. 
A  follow-up  mailing  to  those  who  did  not  respond  to  the  first  mailing  took  place  one  month  after  the  initial 
mailing.  Respondents'  answers  to  the  instrument  remained  anonymous.  Answers  on  the  returned 
questionnaire  were  transferred  to  a  tabulation  sheet. 

After  both  mailings  were  complete,  follow-up  telephone  interviews  were  conducted  with  a  random 
sample  of  10%  of  the  respondents.  Selection  of  this  sample  was  done  by  using  the  numbers  assigned  to 
each  member  of  the  poplation  from  the  original  mailing.  Using  a  random  number  table,  numbers  were 
selected.  If  that  number  corresponds  to  a  member  of  the  population  who  responded  to  the  questionaire,  they 
were  included  in  the  interview.  Selection  continued  until  10%  of  the  respondents  had  been  selected.  If  the 
interviewer  was  not  able  to  reach  the  selected  respondent  after  a  reasonable  period  of  time  due  to  any  variety 
of  uncontrollable  reasons,  new  random  numbers  were  generated  to  replace  those  respondents  who  could  not 
be  reached  by  telephone.  This  process  continued  until  the  number  of  interviews  reached  10%  of  the 
respondent  population.  Responses  to  the  interview  questions  were  recorded  by  the  interviewer  on  paper. 
Each  interview  resulted  in  manual  notations  made  on  questionnaire  sheets  similar  to  Appendix  C. 
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Instrumentation 


The  instrument  used  was  an  adaptation  of  the  instrument  used  in  the  previous  two  studies  (Wertz, 
1984;  Chase,  1993).  The  questions  used  compose  a  test  concerning  three  sections  of  the  Copyright  Act  of 
1976  (Sections  106,  107  and  1 10)  and  the  related  guidelines  (print  materials,  music,  and  off-air  recordings). 
Eight  questions  were  removed  from  the  original  instrument.  These  questions  all  dealt  with  Section  108 
(Limitation  on  exclusive  rights:  Reproduction  by  libraries  and  archives)  and  the  guidelines  for 
photocopying/  interlibrary  loan  arrangements.  The  remaining  questions  were  rewritten  and  expanded  in  an 
attempt  to  make  clear  the  intent  of  the  query.  All  of  these  questions  are  multiple  choice  with  only  one 
correct  response.  Great  effort  has  been  taken  to  clear  any  ambiguity  that  could  have  arisen  from  any  of  the 
questions.  Each  question  and  correct  response  comes  directly  from  the  law  or  the  related  guidelines.  In 
addition,  questions  concerning  demographic  data  were  added  to  the  beginning  of  the  questionnaire,  and  a 
space  for  general  comments  was  added  at  the  end. 

Pilot  Study 

A  pilot  study  was  conducted  using  the  questionnaire.  Volunteers  were  solicited  from  the  Media-L 
Listserv  and  the  CCUMC  (Consortium  of  College  and  University  Media  Centers)  Listserv  on  the  Internet. 
These  lists  are  groups  of  media  professionals  who  exchange  information  via  the  Internet.  A  message  was 
sent  to  each  group  asking  for  help  from  media  directors  at  higher  educational  institutions  who  were  not 
members  of  AECT.  A  total  of  twenty  two  directors  responded  to  the  request  for  assistance.  The 
questionnaire  was  then  transmitted  by  facsimile  to  the  twenty  two  who  indicated  an  interest.  Their  responses 
were  then  returned  by  facsimile.  Twenty  professionals  provided  responses  and  comments.  As  a  direct  result 
of  their  comments,  some  minor  changes  and  revisions  were  made  to  the  instrument.  The  survey  responses 
were  tabulated,  and  the  Office  of  Measurement  and  Evaluation  at  the  University  of  Pittsburgh  provided  an 
analysis  to  determine  the  reliability  of  the  instrument.  Using  the  Kuder-Richardson  20  formula  to  determine 
reliability,  a  score  of  .74  was  achieved.  As  a  self  developed  instrument,  it  was  deemed  reliable. 

Treatment  of  the  Data 

Returned  responses  from  the  study  were  tabulated,  and  the  total  percentage  of  correct  responses  for 
each  respondent  was  determined.  A  total  correct  percentage  score  of  75%  was  used  as  the  criterion  to 
determine  a  proficient  knowledge  level.  This  proficiency  level  is  consistent  with  the  previous  studies 
(Wertz,  1984;  Chase,  1993).  The  total  number  of  respondents  who  achieved  the  75%  proficiency  level  was 
determined.  The  number  of  proficient  respondents  was  divided  by  the  total  number  of  respondents  to 
determine  the  final  percentage  of  proficient  respondents. 

Additional  analysis  took  place  by  examining  the  six  areas  on  which  the  questionnaire  focuses 
(Sections  106,  107,  1 10  &  guidelines  for  classroom  materials,  music,  and  off-air  recordings).  Scores  were 
calculated  that  showed  the  percentage  of  group  members  who  reached  the  75%  proficiency  performance  in 
each  particular  section.  All  of  these  percentage  scores  were  contrasted  to  the  percentage  results  of  the 
previous  studies  (Wertz,  1984;  Chase,  1989). 

The  demographic  data  collected  was  also  examined.  Each  of  the  four  questions  at  the  beginning  of 
the  survey  was  used  to  further  interpret  the  results. 

The  first  question  concerns  the  position  title  and  provides  for  a  response  of:  a)  media  director/ 
media  specialist/  AV  director,  b)  librarians)  faculty,  or  d)  other  (please  explain).  While  one  of  the  primary 
reasons  for  this  question  was  to  ensure  that  respondents  were  members  of  the  population,  the  other  was  to 
attempt  to  establish  the  breakdown  of  media  professionals  who  saw  themselves  as  media  librarians  vs. 
media  "non-librarians." 

The  second  demographic  question  deals  with  the  respondent's  total  number  of  years  experience  as  a 
media  professional.  The  possible  responses  include:  a)  0-5  years,  b)  6-15  years,  c)  16-25  years,  or  d)  25  or 
more  years.  This  question  helped  to  determine  if  new  members  of  the  field  are  better  prepared  to  deal  with 
copyright  questions  than  established  members.  In  contrast,  it  attempted  to  show  that  those  who  have  been 
in  the  field  for  a  number  of  years  appear  to  be  more  competent  when  dealing  with  copyright  related  issues. 

Level  of  education  is  the  third  portion  of  the  demographic  data  collected.  Possible  responses 
include  a)  bachelor's  <!egree,  b)  MLS  master's  degree,  c)  other  mas'  ;r's  degree,  or  d)  doctoral  degree.  Two 
primary  questions  were  evaluated  with  this  question.  First,  did  a  terminal  degree  in  the  field  translate  into  a 
better  understanding  of  copyright  issues?  The  second  question  deals  again  with  the  librarian  vs.  non-librarian 


relationship.  Are  the  MLS  professionals  perhaps  the  best  prepared  of  all  the  media  professionals  when  it 
comes  to  copyright? 

The  final  question  deals  with  the  size  of  the  academic  institution  where  the  respondent  works.  The 
possible  options  include:  a)  under  5000,  b)  5000-10,000,  c)  10,000  -  20,000,  or  d)  over  20,000.  What 
impact  did  the  size  of  the  institution  have  on  director's  knowledge  level  of  copyright?  Do  members  of  large 
institutions  have  a  more  in  depth  knowledge  of  copyright  or  are  there  specialists  on  campus  who  relieve 
them  of  that  responsibility? 

Once  all  the  data  was  collected,  an  evaluation  was  made  to  determine  the  results.  The  first 
evaluation  examined  all  of  the  respondents  who  achieved  or  exceeded  tlie  75%  criterion.  The  demographic 
responses  of  only  the  proficient  group  were  tabulated  and  contrasted. 

The  second  evaluation  examined  the  percentage  of  respondents  who  reached  the  proficient  state  in 
each  of  the  six  divisions.  These  scores  are  contrasted  to  the  scores  of  the  1984  and  1993  study. 

Finally,  a  Chi-square  test  was  used  to  examine  the  relationship  between  demographic 
characteristics  and  proficiency.  Each  of  the  four  demographic  areas  were  tested  in  an  attempt  to  identify 
significant  scores  in  any  particular  group.  This  was  done  by  comparing  expected  percentages  of  proficient 
respondents  to  actual  percentage  of  proficient  respondents  to  each  of  the  four  demographic  responses.  The 
final  result  produced  a  statistical  table  for  each  of  the  four  demographic  questions.  Each  table  shows  the 
percentage  of  proficient  and  non-proficient  respondents  within  each  of  the  possible  four  choices  in  the 
demographic  question.  This  test  was  extended  by  cross  tabulating  the  six  areas  of  the  law,  demographics, 
and  proficiency  to  determine  levels  of  significance. 

Questionnaire  Responses 

On  May  13,  1994,  466  copies  of  the  questionnaire  (Appendix  A)  along  with  a  cover  letter 
(Appendix  B)  and  a  self-addressed,  stamped  envelope  were  mailed  to  identified  AECT  members.  On  June  1 5, 
1994,  266  follow-up  letters  with  another  copy  of  the  questionnaire  and  another  self-addressed,  stamped 
envelope  were  mailed  to  those  individuals  who  had  not  yet  responded.  By  July  15,  1994,  two  hundred 
members  had  responded  by  completing  the  questionnaire.  Forty-five  others  had  mailed  back  the  cover  letter 
indicating  that  they  were  not  part  of  the  population.  Seven  individuals  indicated  that  they  did  not  wish  to 
participate.  Seven  letters  were  returned  by  the  post  office  as  "undeliverable,"  and  one  member  of  the 
population  died. 

Survey  Results 

The  responses  to  all  the  survey  questions  were  compiled  and  analyzed.  The  reliability  of  the 
in?  jument  was  again  calculated.  Using  the  Kuder-Richardson  20  formula,  a  score  of  .79  was  realized.  The 
instrument  was  again  deemed  reliable. 

The  mean  score  of  the  respondents  on  the  instrument  was  58.6%.  Eighteen  percent  (or  36)  of 
respondents  scored  75%  or  higher  on  the  instrument.  This  contrasts  to  15%  in  the  1984  study  and  10%  in 
the  1993  study.  A  complete  breakdown  of  percentages  of  respondents  who  reached  the  proficient  state  of 
75%  or  higher  in  each  of  the  different  instrument  sections  is  shown  in  Table  1. 

The  section  with  highest  percentage  of  respondents  reaching  the  proficient  state  was  the  Guidelines 
for  Off-Air  Videotaping  with  62%.  The  section  with  the  lowest  percentage  of  respondents  reaching  the 
proficient  state  was  Section  1 10  (Limitations  on  Exclusive  Rights:  Exemptions  of  Certain  Performances 
and  Displays)  with  only  26%.  Three  of  the  sections  showed  higher  percentages  than  either  of  the  previous 
two  studies.  These  included  Section  106  (Exclusive  Rights),  Section  1 10,  and  the  Guidelines  for  Classroom 
Copying  with  Respect  to  Books  and  Periodicals.  Only  Section  107  (Limitations  on  Exclusive  Rights:  Fair 
Use)  showed  a  lower  percentage  than  the  previous  two  studies.  Two  of  the  sections  showed  mixed 
results  with  the  percentages  falling  between  the  previous  two  studies.  These  included  the  Guidelines  for 
Classroom  Use  of  Music  and  the  Guidelines  for  Off-Air  Recording  of  Broadcast  Programming  for 
Educational  Purposes. 
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Table  1 


Percentage  of  Respondents  Who  Reached  the  Profident  State  in  Each  Division 


Area  of  Copyright 

%  in  1984  Study 

%  in  1993  Study 

%  in  1994  Study  using 

Law 

using  1984  instrument 

revised  instrument 

(n=144) 

(N=93) 

(n=200) 

Section  106 

47% 

53% 

61% 

Exclusive  Rights 

(n=68) 

(n=49) 

(n=122) 

Section  107 

71% 

66% 

56% 

Fair  Use 

(n=102) 

(n=61) 

(n=112) 

Section  110 

11% 

8% 

26% 

Public  Performance 

(n=16) 

(n=7) 

(n=52) 

Classroom 

23% 

14% 

31% 

Guidelines 

(n=33) 

(n=13) 

(n=62) 

Musk 

32% 

25% 

28% 

Guidelines 

(n=46) 

<n=23) 

(n=56) 

Off-Air  Videotaping 

59% 

74% 

62% 

Guidelines 

(n=85) 

(n=69) 

(n=124) 

All  sections 

15% 

10% 

18% 

(n=21) 

(n=9) 

(n=36) 

Table  2 


Number  of  Proficient  Respondents  Separated  by  Demographic  Response? 


Ouestion 

Responses 

Position 

(media  director) 
30 

(librarian) 
1 

(faculty) 
0 

(other) 
5 

Years  in 
Profession 

(0-5  years) 
2 

(6-15  years) 
14 

(16-25  years) 
18 

(25  or  more) 
2 

Highest  Degree 
Earned 

(bachelor's) 
2 

(MLS) 
3 

(master's) 
18 

(doctoral) 
13 

Size  of  Institution 

(<5000) 
13 

(5-10,000) 
9 

(10-20,000) 
6 

(>20,000) 
8 

(n=36) 

Table  2  shows  the  breakdown  of  how  the  proficient  respondents  answered  the  demographic 
questions.  A  total  of  36  respondents  scored  75%  or  higher  on  the  survey.  The  proficent  respondents' 

demographic  responses  are  indicated  below.  Each  row  of  responses  will  total  36  for  the  number  of 
proficient  respondents.  It  should  be  noted  that  each  column  did  not  have  an  equal  number  of  respondents. 
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Further  evaluation  was  done  on  the  demographic  data  in  an  attempt  to  identify  significant 
differences.  A  Chi-Square  analysis  was  done  to  cross  tabluate  each  of  the  demographic  data  areas  with 
proficiency.  A  significance  level  of  .05  was  used  to  interpret  the  results.  Tables  3  through  6  show  the 
results  of  that  analysis. 


Table  3 

Cross  Tabulation  of  Position  Title  with  Proficiency 

Response 

Proficient 

fo 
(fe) 

Up.  Profit 

(fe) 

media  director 

30 
(29.3) 

133 
(133.7) 

librarian 

1 

(1.3) 

6 
(5.7) 

faculty 

0 

(.9) 

5 
(4.1) 

other 

5 
(4.5) 

20 
(20.5) 

fo  =  frequency  observed 
fe  =  frequency  expected 

n  =  200 
X1  =  .74131 

Table  4 

Cross  Tabulation  of  Years  of  Experience  with  Proficiency 

Response 

Proficient 
fo 
(fe) 

Not  Proficient 

fo 
(fe) 

0-5 

2 

(4.1) 

21 
(18.9) 

6-15 

14 

(15.3) 

71 
(69.7) 

16-25 

18 

(13.0) 

54 
(59.0) 

25  or  more 

2 

(3.6) 

18 
(16.4) 

fo  =  frequency  observed 
fe  =  frequency  expected 

n  =  200 
XJ=  .19176 
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Table  5 


Crros  Tabulation  of  Highest  Level  of  Education  vn*V>  PmfiHt»npy 


Response 

Proficient 

fo 
(fe) 

Not  Proficient 

fo 
(fe) 

bachelor's 

2 

(3.9) 

20 
(18.1) 

MLS 

3 

(3.7) 

18 

(17.3) 

other  master's 

17 

(17.2) 

80 
(79.8) 

doctoral 

13 

(10.1) 

44 

(46.9) 

fo  =  frequency  observed 
fe  =  frequency  expected 

n  =  197* 
X*=  .51202 
*  3  responses  could  not  be  used 

Table  6 

Ons*  Tabulation  nf  Tnstlhirtnn  Full  Time  Enm11m<»nt  (FTE)  with  Profidtncv 

FTE 

Proficient 

fo 
(fe) 

Not  Proficient 

fo 
(fe) 

<5000 

13 
(12.8) 

58 
(58.2) 

5000-10,000 

9 

(7.9) 

35 
(36.1) 

10,000  -  20,000 

6 
(7.2) 

34 
(32.8) 

>  20,000 

8 

(8.1) 

37 
(36.9) 

fo  =  frequency  observed 
fe  =  frequency  expected 

n=200| 
X*=  .93406| 

Each  of  the  four  cross  tabulations  showed  a  level  of  significance  of  >.05.  Therefore,  the  number  of 
proficient  and  non-proficient  respondents  in  each  of  the  demographic  categories  can  be  considered  not 
significant.  None  of  the  areas  showed  an  unusually  high  or  low  number  of  respondents. 

Each  of  the  six  areas  of  the  law  were  also  contrasted  with  the  demographic  data  in  relationship  to 
proficiency.  The  respondents  who  reached  the  criterion  level  of  proficiency  in  the  specific  section  were 
contrasted  with  their  demographic  responses.  Only  one  section  showed  a  level  of  significance  <  .05.  Table  7 
shows  the  analysis  for  the  cross  tabulation  of  the  full  time  enrollment  on  Section  107  and  proficiency. 
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Table  7 


Cross  Tabulation  of  FuU  Time  Enrollment  by  Section  107  with  Proficiency 


Proficient 

Not  Proficient 

fo 

/° 

(fe) 

f/e) 

<  5000 

43 

28 

(39.4) 

(31.6) 

5000-10,000 

16 

28 

(24.4) 

(19.6) 

10,000  -  20,000 

25 

15 

(22.2) 

(17.8) 

>  20,000 

27 

18 

(25.0) 

(20.0) 

fo  =  frequency  observed 

n  =  200 

fe  =  frequency  expected 

X'=  .03802 

It  can  be  determined  that  there  are  some  significant  differences  in  the  results  of  Table  7.  Most 
notably  is  the  poor  performance  by  members  of  the  5000  -  10,000  group,  who  showed  only  16  proficient 
members  when  questioned  about  Section  107  (fair  use).  The  other  three  groups  all  showed  a  higher  than 
anticipated  number  of  proficient  respondents. 

At  the  end  of  the  survey  a  place  was  provided  for  comments.  Almost  a  third  of  the  respondents 
chose  to  make  comments  concerning  the  study  or  on  copyright  in  general.  The  comments  ranged  from 
general  observations  about  copyright  to  specific  requests  for  information.  Many  people  asked  for  a  copy  of 
the  findings,  the  correct  answers,  or  suggestions  of  good  reference  materials  dealing  with  copyright.  Many 
of  these  requests  have  been  filled.  However,  the  responses  were  blind,  and  if  the  respondent  did  not  provide 
an  address  or  name,  it  was  impossible  to  do  so.  Others  requested  that  the  results  be  posted  on  the  Internet 
for  review. 

Several  comments  were  made  concerning  multimedia,  computer  software,  and  distance  learning 
issues  and  their  noted  absence  in  the  survey.  A  number  of  people  also  noted  that  some  of  the  subject  areas 
were  not  relevant  to  their  job  area.  As  a  result,  these  people  were  also  concerned  that  their  scores  would  not 
represent  their  actual  copyright  knowledge  level  within  their  position. 

A  few  people  critiqued  the  methodology  of  the  study  asking  if  the  questions  would  really  provide 
any  answers.  Two  individuals  noted  that  this  was  an  exam  not  a  survey  or  complained  about  the  length  and 
depth  of  the  quesionaire.  One  director's  response  summed  up  the  negative  comments: 

This  is  much  more  of  a  quiz  on  whether  the  reader  did  his  or  her  reading 
assignment  than  it  is  an  attempt  to  survey  the  working  knowledge  of 
practioners.  You've  been  taking  graduate  courses  too  long.  You  haven't 

found  our  how  "knowledgeable  I  am  about .  . ."  You  found  out  how  good  I 
am  at  taking  a  confusing  quiz. 

These  sentiments  were  not  shown  by  the  majority  of  respondents.  Many  of  the  comments  were 
positive  and  noted  the  importance  of  copyright  in  the  profession.  Some  members  applauded  the  effort  to  do 
research  on  such  a  timely  and  controversial  topic.  Others  mentioned  that  after  attempting  to  answer  the 
questions  that  they  realized  they  weren't  as  familiar  with  the  law  as  they  thought  they  were.  One  respondent 
commented  that  copyright  needed  to  be  included  in  education  methods  courses  and  not  left  only  to  media 
professionals. 
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Follow-up  Interviews 


During  the  last  two  weeks  in  July,  follow-up  interviews  were  conducted  with  a  random  sample  of 
10%  of  the  respondents.  During  the  interview  process,  it  was  necessary  to  replace  four  members  of  the 
interview  group.  Two  members  could  not  be  reached.  One  member  was  on  extended  leave,  and  the  fourth 
was  no  longer  with  the  institution.  Their  replacements  were  selected  using  the  same  process  as  the  initial 
selections. 

Each  interview  began  with  an  introduction  refreshing  their  memory  about  the  study  and  a  reminder 
that  all  of  their  responses  would  remain  anonymous.  The  first  question  determined  if  they  felt  the  questions 
provided  an  accurate  measurement  of  their  copyright  knowledge.  Fourteen  of  the  twenty  subjects  indicated 
that  they  thought  the  questions  provided  an  accurate  measurement  of  their  copyright  knowledge.  Some 
members  commented  that  they  felt  the  questions  were  "broad  based"  and  "appropriate,"  although  others  who 
answered  yes  also  noted  that  some  of  the  questions  were  not  related  to  their  specific  work.  While  one 
subject  could  not  recall  the  questions,  five  indicated  that  they  did  not  feel  the  questions  provided  an  accurate 
measurement.  Their  comments  noted  that:  they  knew  more  than  they  were  asked,  they  were  unhappy  with 
the  question  structure,  they  felt  the  questions  were  somewhat  skewed  and  did  not  show  enough  concern  for 
multimedia  applications,  or  that  their  position  dealt  primarily  with  print  materials  and  the  majority  of  the 
questions  were  inappropriate. 

The  second  question  determined  if  there  were  areas  of  the  law  that  should  have  been  added  or 
removed  from  the  questionnaire.  None  of  the  subjects  mentioned  any  areas  that  they  thought  should  be 
removed.  There  were  a  number  of  suggestions  for  additions.  Three  members  indicated  multimedia  should 
have  been  included.  Two  members  noted  computer  software  and  distance  education  issues.  Other  topics 
discussed  the  inclusion  of  more  on  print  materials,  suggested  guidelines,  videotaping,  format  shifting,  new 
media,  and  fair  use  material. 

Question  number  three  wanted  to  know  if  the  questions  on  the  instrument  were,  or  were  not, 
relevant  to  their  job  at  the  institution.  Two  of  the  subjects  indicated  that  most  of  the  material  was  irrelevant 
to  their  job.  The  music  guidelines  and  the  print  guidelines,  each  received  five  votes  as  irrelevant.  Off-air 
videotaping  was  also  mentioned  once  as  not  being  relevant.  The  remainder  did  not  provide  a  response. 

The  fourth  question  attempted  to  discover  how  much  help  participants  were  getting  in  answering 
the  questions.  Did  they  use  any  reference  materials  or  seek  assistance  in  answering  any  of  the  questions?  If 
they  did,  from  what  sources  and  how  much?  Four  of  the  interviewees  indicated  that  they  did  use  some 
materials  in  answering  the  questions.  Two  of  them  noted  that  they  had  a  copy  of  the  law  and  referred  to  it. 
One  mentioned  a  publication  by  Gary  Becker  entitled  ©  Copyright:  A  Guide  to  Information  and  Resources, 
and  another  mentioned  a  a  publication  by  AIME  entitled  A  Viewer's  Guide  to  the  Copyright  Law:  What 
Every  School.  College,  and  Public  Library  Should  Know. 

Question  number  five  inquired  into  the  subjects'  background,  and  where  and  how  they  had  obtained 
their  copyright  training.  Professional  journals  was  the  most  often  cited  source  with  ten  members 
mentioning  them.  Nine  subjects  indicated  that  they  had  attended  a  seminar  or  workshop  on  copyright.  Seven 
indicated  that  they  had  attended  at  least  one  session  at  the  AECT  national  conference  that  related  to 
copyright.  Seven  also  mentioned  books  and  publications  that  they  had  read  to  gain  experience.  Three 
indicated  that  their  formal  higher  education  had  included  copyright  issues  as  part  of  the  course  work.  Two 
mentioned  the  Internet  and  listservs  as  a  source  for  information,  and  finally  one  had  watched  a  teleconference 
on  copyright  issues  in  education. 

The  sixth  question  was  a  broad  question  asking  what  should  a  media  professional  know  about 
copyright.  The  most  common  and  flippant  answer  was  "as  much  as  possible,"  but  many  went  on  to  cite 
specific  areas.  These  included  the  guidelines  (3  members),  computer  software  (4  members),  distance  learning 
(3  members),  duplication  of  materials  (2  members),  and  fair  use  (3  members).  The  rest  of  the  answers  were 
a  hodgepodge  of  suggestions  and  insights.  These  included:  how  to  reach  faculty  with  copyright  information, 
what  was  permissible,  how  to  stay  out  of  trouble,  ability  to  clearly  explain  the  law,  be  too  conservative, 
know  the  current  issues,  know  what  faculty  can  and  can't  do,  have  access  to  legal  help,  know  who  the 
definitive  resource  is  on  your  campus,  and  applications  to  student  projects.  One  member  summed  up  the 
question  by  stating  that  we  simply  need  for  a  code  of  ethics  in  our  profession  and  for  media  professionals  to 
use  it  in  their  work. 

The  seventh  question  attempted  to  gain  insight  on  how  the  administration  at  each  institution 
regarded  copyright  issues.  Twelve  of  those  people  interviewed  described  their  administration  as  viewing 
copyright  as  an  important  or  very  important  issue.  Two  people  described  their  actions  as  very  good  on 
computer  software  issues,  but  nonexistent  for  other  non-print  media.  Four  people  said  their  institution 
tended  to  ignore  copyright  issues,  and  two  described  their  institution  as  ambivalent. 
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Question  number  eight  provided  the  members  an  opportunity  to  comment  on  AECT's  effort  to 
provide  copyright  information.  Seventeen  of  the  interviewees  reacted  positively  noting  the  publications  and 
efforts  of  the  Association.  One  described  their  efforts  as  acceptable,  while  one  claimed  they  had  not  seen 
much  from  AECT.  Another  interviewee  described  their  efforts  negatively.  They  described  a  recent  copyright 
initiative  with  CCUMC  that  the  copyright  committee  of  AECT  had  pledged  to  support  and  then  had  not 
followed  through. 

The  ninth  question  was  concerned  with  how  media  professionals  could  best  provide  copyright 
information  to  other  media  professionals.  Eight  members  mentioned  professional  journals  as  an  avenue  for 
the  distribution  on  information.  Six  noted  the  Internet,  with  some  suggesting  a  listserv  for  copyright 
issues.  Print  materials  for  distribution  were  mentioned  by  five  respondents.  Four  listed  workshops  or 
conferences  as  one  of  the  best  ways  to  gt,t  the  word  out.  Two  people  mentioned  the  possibility  of 
teleconferences,  and  one  suggested  a  videotape  for  distribution.  Another  insisted  that  copyright  needs  to  be 
part  of  every  related  academic  program  and  taught  accordingly.  Finally,  one  member  suggested  the  best  way 
was  by  setting  a  good  example. 

The  final  question  provided  an  opportunity  for  any  final  comments  that  professionals  wanted  to 
make  about  the  study  or  about  copyright  in  education.  Some  who  responded  directly  about  the  study 
mentioned  that  they  felt  the  study  was  either  important,  valuable  or  worthwhile  and  appreciated  someone 
doing  the  research.  Two  asked  specifically  to  be  sent  the  results,  while  one  mentioned  that  they  were  fearful 
of  the  results.  One  suggested  that  the  study  should  have  been  more  flexible  allowing  greater  leniency  to 
members  in  areas  that  were  not  appropriate  to  their  work. 

With  regard  to  copyright  in  general,  two  members  mentioned  the  difficulty  of  the  copyright  issues 
surrounded  new  technologies  in  multimedia,  distance  learning,  and  electronic  publishing.  One  interviewee 
discussed  the  difficulty  of  reaching  the  faculty  and  educating  them  to  the  importance  of  copyright. 


Discussion  &  Findings 

As  predicted,  a  small  percentage  of  media  directors  were  able  to  reach  the  proficiency  level  of  75%. 
Only  18%  (as  compared  to  15%  in  the  1984  study)  were  able  to  do  so.  This  despite  efforts  to  revise  the 
instrument  to  make  it  more  readable  and  the  elimination  of  two  sections  considered  irrelevant  to  the  work  of 
many  media  directors.  The  mean  score  for  the  respondents  was  computed  at  58.6%,  demonstrating  that  the 
directors  did  seem  to  at  least  have  some  background  in  copyright.  This  is  higher  than  the  mean  score  from 
the  1993  study  which  was  56.6%  (The  mean  score  from  the  1984  study  is  not  available).  If,  as  suggested  by 
some  respondents,  there  had  been  a  response  of  "not  applicable"  for  those  questions  that  were  not  relevant  to 
the  subject's  work,  we  could  have  expected  the  number  of  proficient  subjects  and  the  mean  scores  to  rise. 

The  survey  method  of  data  collection  must  be  scrutinized  when  reviewing  the  results.  The  system 
developed  to  survey  the  directors  leaves  some  degree  of  ambiguity  in  interpreting  the  results,  regardless  of 
how  carefully  it  was  administe/ed.  The  attempt  of  the  study  was  to  identify  the  level  of  knowledge  that 
directors  had  and  use  in  their  work  environment.  It  was  anticipated  that  some  directors  would  use  reference 
materials  or  seek  assistance  in  answering  the  questions.  The  cover  letter  spoke  directly  to  this  by  stating 
"provide  responses  to  these  questions  using  the  same  procedure  as  you  would  under  normal  working 
conditions."  If  reference  materials  or  subject  experts  were  available  and  regularly  used  by  the  professional, 
then  they  should  be  available  for  use  in  answering  the  survey.  The  objective  was  not  to  test  the  subject's 
memorization  but  to  determine  the  quality  of  "working"  information  that  directors  were  providing  to  their 
faculty  and  staff.  The  mailed  survey  can  only  give  one  perspective  of  this  knowledge  level.  Other 
instruments  may  identify  different  percentages,  but  it  is  predicted  that  those  percentages  would  not  deviate 
significantly  from  the  results  of  this  study. 

During  the  follow-up  interviews,  20%  of  the  directors  indicated  that  they  sought  assistance  in 
answering  the  questions.  Though  the  possibility  of  more  respondents  examination  of  the  copyright  law  and 
its  guidelines  might  exist  in  their  attempt  at  answering  the  questionaire.  The  researcher  considered  these 
respondents'  efforts  in  this  respect  as  a  positive  sincere  attempt  on  their  part  at  finding  out  what  the  law  and 
its  guidelines  provide  in  answers  to  the  questions  at  hand.  The  figure  of  20%  is  surprisingly  close  to  the 
percentage  of  proficient  respondents.  The  correlation  of  the  two  was  not  examined,  but  future  studies  might 
find  a  relationship.  Perhaps  one  of  the  major  outcomes  of  this  study  may  be  that  many  directors  locate  and 
use  reference  materials.  A  number  of  people  commented  at  the  end  of  the  questionnaire  that  they  were  not  as 
familiar  with  the  law  as  they  thought,  or  that  they  needed  to  review  some  of  the  materials.  Several 
respondents  asked  about  a  definitive  resource  for  copyright  issues  in  education.  Suggestions  were  mailed 
back  to  those  who  requested  the  information. 
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The  increase  in  the  total  proficiency  of  the  group  to  1 8%,  while  still  low,  is  an  encouraging  sign. 
The  trend  is  moving  in  a  positive  direction  toward  a  more  informed  group.  The  current  study  had  over  200 
respondents,  almost  twice  the  number  of  the  previous  studies.  The  survey  provided  language  that  worked 
toward  discovering  the  respondent's  real  knowledge  level  and  not  their  ability  to  decipher  the  question. 
However,  there  was  still  criticism  from  the  respondents  concerning  the  difficulty  and  apparent  trivial  nature 
of  some  of  the  questions. 

The  individual  section  scores  showed  some  interesting  trends.  Sections  106,  110,  and  the 
classroom  print  guidelines  all  showed  the  highest  percentage  of  proficient  respondents  of  the  three  studies. 
Copyright  continues  to  be  an  issue  discussed  regularly  at  professional  association  meetings  and  in 
professional  publications.  This  awareness  may  have  partially  manifested  itself  in  an  understanding  of  the 
basic  rights  of  a  copyholder  (Section  106).  This  would  explain  the  percentage  rise  for  Section  1 06.  The  rise 
in  Section  1 10  is  probably  primarily  related  to  the  rewording  of  some  of  the  questions  from  this  area.  Those 
questions  in  particular  were  difficult  to  read  because  of  the  legal  language.  The  restructured  questions 
perhaps  gave  a  better  insight  into  the  actual  understanding  of  the  Section  by  the  directors.  Another 
explanation  for  the  increase  could  be  the  recent  reexamination  of  the  Section  given  the  new  issues  arising  in 
distance  education.  Debate  continues  on  the  merits  of  public  performance  exemptions  on  distance  learning 
applications.  These  issues  may  have  aided  in  supplementing  the  professionals  knowledge  of  this  area  of  the 
law.  The  classroom  guidelines  for  print  materials  also  saw  its  highest  percentage  of  proficient  respondents. 
No  specific  event  can  be  attributed  to  this  increase.  However,  some  of  these  questions  were  also  rewritten, 
and  perhaps  the  attention  generated  by  the  recent  Kinko's  case  has  brought  additional  awareness  to  these 
issues. 

The  decrease  of  proficiency  in  Section  107  (fair  use)  is  the  most  intriguing  of  the  results  and  may 
be  the  most  difficult  to  specifically  explain.  The  other  two  sections  of  the  1976  law  both  showed  increases 
in  the  percentage  of  proficient  respondents.  Fair  use  continues  to  be  one  of  the  most  misunderstood  and 
misinterpreted  portions  of  the  law  (Sinofsky,  1984).  This  is  one  of  the  areas  where  only  a  little  knowledge 
can  be  damaging,  since  a  number  of  variables  impact  on  fair  use  interpretations.  While  the  majority  (88%) 
understood  that  fair  use  was  applicable  to  teaching,  news  reporting  and  criticism,  only  36%  knew  that  the 
medium  was  not  a  factor  when  determining  fair  use.  When  asked  what  the  key  to  the  purpose  and  character 
of  fair  use  was,  63%  were  able  to  identify  the  correct  response  "for  non-profit  educational  purposes." 
Seventy-five  percent  identified  the  effect  of  the  use  of  a  copyrighted  work  upon  the  potential  market  as  a 
"major  consideration."  Based  on  these  responses  it  would  appear  that  many  are  familiar  with  fair  use,  but 
could  not  accurately  identify  and  evaluate  specific  applications  using  the  four  criteria  for  fair  use.  This  is  a 
difficult  area  of  the  law  for  media  directors  and  is  the  one  area  that  specifically  shows  the  need  for  additional 
instruction.  Many  directors  are  quick  to  claim  an  understanding  of  fair  use,  but  few  can  probably  recall  the 
four  criteria  for  the  application  of  fair  use. 

Both  the  Music  and  Off-Air  Videotaping  Guidelines  received  mixed  results.  Their  percentages  fell 
between  the  previous  two  studies.  Several  of  the  respondents  noted  that  one  or  both  of  these  areas  were  not 
applicable  to  their  work.  It  should  be  noted  that  17%  of  the  respondents  did  not  answer  each  of  two  of  the 
music  questions.  Changes  in  technology  have  also  complicated  the  issues  of  videotaping  programs.  Many 
directors  (33.5%)  did  not  understand  that  cable  television  programming  is  not  covered  by  the  off-air 
guidelines.  Additional  instruction  is  required  for  both  areas. 

The  attempt  to  cross  tabulate  the  demographic  statistics  with  proficiency  provided  no  significant 
findings.  The  first  statistic  involved  the  analysis  of  position  and  proficiency.  As  discussed  earlier,  this 
question  was  primarily  a  filter  to  eliminate  respondents  who  were  not  part  of  the  population.  One 
respondent  was  eliminated  when  she  indicated  she  was  a  media  professional  for  a  secondary  school  district. 
The  media  director/specialist  choice  was  selected  by  8 1 .5%  of  the  respondents.  It  was  therefore  difficult  to 
draw  any  conclusions  due  to  the  low  number  of  responses  for  the  other  choices.  The  Chi- square  analysis 
found  no  significant  difference  in  the  number  of  proficiencies  in  each  of  the  selections. 

The  cross  tabulation  of  years  in  the  profession  with  proficiency  showed  no  significance  at  the  .05 
level,  but  it  was  interesting  to  note  which  group  had  the  highest  number  of  proficient  respondents.  The 
group  who  had  been  employed  1 6  -  25  years  composed  36%  of  the  total  group,  yet  they  were  50%  of  the 
total  number  of  proficient  subjects.  The  X2  was  .19  meaning  that  statistically  this  could  have  happened  by 
chance,  but  future  studies  may  examine  this  area  and  find  a  discrepancy. 

The  last  two  cross  tabulations  with  total  proficiency  (highest  degree  earned  and  size  of  institution) 
showed  no  irregularities.  Their  X2  s  were  .5 1  and  .93  indicating  that  the  results  are  consistent  with  the 
population. 

The  only  significant  result  was  found  when  the  six  individual  sections  of  the  copyright  law  were 
cross  tabulated  with  the  demographics  and  section  proficiency.  When  full  time  enrollment  and  Section  1 07 


(fair  use)  were  analyzed  with  the  proficiency  levels,  the  level  of  significance  was  only  .03.  Members  who 
worked  at  institutions  that  had  a  full  time  enrollment  of  5000  -  10,000  scored  significantly  lower  than 
anticipated.  It  is  difficult  to  explain  the  poor  performance  of  this  specific  group  since  their  overall 
proficiency  level  was  not  below  expectations.  There  are  misconceptions  concerning  the  understanding  and 
applications  of  fair  use  as  discussed  earlier  and  in  chapter  two.  This  group  appears  more  confused  on  this 
issue  than  most,  and  prooably  doesn't  have  legal  counsel  on  campus  to  provide  regular  interpretations.  They 
are  apt  to  be  making  fair  use  judgement  at  their  own  discretion.  Directors  need  to  fully  understand  fair  use  in 
order  tu  apply  it  properly. 

The  attitudes  of  the  directors  were  apparent  from  the  comments  made  on  the  questionnaire  and  from 
the  follow-up  interviews.  Over  half  of  the  population  returned  the  instrument  in  some  form.  Given  the  time 
and  effort  required  to  complete  the  questionnaire,  copyright  must  be  an  important  issue  for  media  directors 
to  invest  that  much  effort.  This  became  further  apparent  when  many  respondents  attempted  to  justify  their 
shortcomings  at  the  end  of  the  survey.  Many  expressed  the  importance  of  the  issue  and  knew  they  that 
needed  to  be  well  informed. 

Each  of  the  directors  who  were  interviewed  had  received  or  participated  in  some  type  of  copyright 
training.  Many  spoke  of  workshops,  reference  materials,  orconference  activities  as  the  source  for  the 
insight  they  had  gained  on  the  subject.  None  of  the  interviewees  were  apathetic  or  ambivalent  to  the  issues 
of  copyright  in  education.  In  fact,  the  reverse  was  true.  Most  of  the  directors  had  a  specfic  point  of  view  that 
they  felt  was  important  to  convey.  Often  this  position  was  expressed  as  soon  as  they  were  asked  the  first 
question.  Often  this  position  was  a  recurring  theme  in  the  answers  to  many  of  their  questions  (i.e.  new 
technologies  are  making  copyright  more  difficult,  educating  our  faculty  is  the  most  difficult  aspect,  our 
institution  does  not  see  this  as  an  important  issue,  overly  conservative  interpretations  of  fair  use  will 
eventually  hurt  us  all). 

As  might  be  expected  there  were  no  clear  solutions  provided  by  the  interviews.  The  concensus 
agreed  that  we  needed  to  keep  working  to  assure  educators  are  provided  representation  in  the  modeling  of 
new  guidelines  and  in  the  creation  of  future  copyright  acts.  While  the  results  of  the  mailed  survey  showed  a 
minor  percentage  of  directors  at  the  competency  level,  the  interviews  and  the  comments  indicated  that 
directors  are  aware  of  the  issues  and  what  is  at  stake.  Copyright  is  a  difficult  issue  to  manage  on  college 
campuses.  The  struggle  to  maintain  ethical  practices  and  work  within  institutional  budgets  sometimes 
causes  a  strain  on  middle  managers,  like  media  directors.  They  are  caught  between  the  faculty's  request  for 
services  that  may  border  on  infringement,  and  a  budget  that  cannot  provide  a  purchase  request  for  needed 
materials  to  provide  instruction.  A  clear  policy  instituted  by  a  knowledgable  director  is  the  best  way  to 
provide  equity  to  all  the  parties  involved. 

Conclusions 

The  results  of  this  study  demonstrate  that  the  majority  of  media  directors  across  the  country  do  not 
have  a  proficient  knowledge  of  the  copyright  law  and  related  guidelines.  A  meager  1 8%  were  able  to  achieve 
the  established  proficiency  level  of  75%.  All  of  the  questions  that  they  were  asked  came  directly  from  the 
law  and  related  guidelines.  The  study  intentionally  eliminated  any  questions  that  might  have  an  ambiguous 
answer  or  would  cause  debate  over  interpretation. 

The  positive  result  from  these  findings  is  that  the  percentages  appear  to  be  moving  upward.  The 
1 8%  of  proficient  respondents  is  the  highest  level  achieved  i  n  any  of  the  previous  studies.  The  follow-up 
interviews  also  demonstrated  that  there  is  an  acute  awareness  of  copyright  in  the  media  profession  in  higher 
education.  Media  directors  may  not  always  have  the  correct  answer  to  copyright  questions,  but  they  will 
usually  have  an  opinion.  Almost  all  of  the  directors  mentioned  some  type  of  professional  development  that 
they  had  done  concerning  copyright.  Some  of  the  work  had  been  voluntary,  but  others  described  mandatory 
sessions  that  they  attended  at  a  local  or  state  level. 

This  study  focused  on  members  of  AECT,  the  largest  professional  organization  for  media 
professionals.  The  organization  maintains  a  copyright  committee  that  works  to  educate  and  inform  the 
group  members.  It  should  be  noted  that  the  results  of  this  study  showed  the  positive  influence  that  the 
group  is  beginning  to  have.  Many  of  the  members  referred  to  the  column  the  committee  sponsors  in  the 
association's  publication,  Tech  Trends.  When  asked  about  the  efforts  of  AECT  in  providing  copyright 
information,  90%  of  the  directors  responded  with  a  "very  good"  or  "excellent".  They  were  familiar  with  the 
workshops  the  group  had  conducted  and  the  sessions  they  had  sponsored.  The  lone  negative  response  was 
from  a  member  who  was  upset  over  the  association's  lack  of  support  for  an  initiative  that  another 
organization  was  sponsoring.  Further  investigation  into  the  situations  revealed  that  AECT  had  provided 
considerable  support  and  representation  to  the  initiative. 
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The  work  of  AECT  needs  to  continue  and  be  expanded.  They  have  at  hand  a  critical  issue  that  is  at 
a  highly  vulnerable  stage  of  development.  The  issues  in  copyright  are  remerging  in  the  continued 
development  of  technology.  As  this  work  goes  to  press,  the  presidential  commission  on  the  National 
Information  Infrastructure  QUI)  Advisory  Council  is  preparing  a  document  to  begin  to  deal  with  many  of 
the  issues  brought  on  by  the  information  highway.  One  of  those  issues  will  be  copyright.  The  report  from 
that  council  will  be  the  blueprint  for  the  structure  of  future  copyright  legislation  and  guideline  development. 
AECT  can  take  the  leadership  role  by  recognizing  these  developments  and  championing  the  causes  of  its 
constituents. 

Recommendation-;  for  Improvement     Future  Study 

The  educating  of  media  directors  needs  to  continue.  The  study  results  show  that  there  are  some 
obvious  knowledge  gaps  in  media  directors'  backgrounds.  In  particular,  the  interpretation  of  fair  use  is  still 
a  misunderstood,  and  misapplied  concept,  and  should  receive  the  greatest  concentration.  The  content  areas 
concerning  the  music  and  off-air  guidelines  also  need  more  consideration  or  instructional  emphasis. 
However,  the  scores  show  that  there  actually  is  no  area  of  the  law  in  which  the  media  directors  can 
demonstrate  a  competency. 

The  dissemination  of  information  needs  to  continue  from  professional  organizations,  government 
bodies,  and  other  concerned  parties.  ABCT's  efforts  were  mentioned  favorable  by  almost,  all  of  the 
interviewees.  AECT' s  publication  Tech  Trends  features  a  regular  column  on  copyright.  Their  efforts  should 
be  continued  and  expanded.  The  AECT  copyright  committee  is  currently  considering  sponsoring  a 
teleconference.  Two  copyright  teleconferences  have  been  aired  recently,  but  both  were  targeted  at  different 
audiences.  Such  a  venture  would  reach  many  professionals.  New  avenues,  like  teleconferences,  need  to  be 
continually  developed  in  attempting  to  reach  practior.ers  with  information. 

New  questions  are  continually  being  asked  as  technology  changes  and  the  answers  to  old  questions 
are  constantly  being  reevaluated.  The  debate  and  disscussion  of  these  issues  is  important  if  we  are  to  come 
to  any  resolutions.  This  should  take  place  through  a  variety  of  methods  including  most  of  those  currently 
being  used  like  professional  journals,  conference  sessions,  and  workshops.  One  of  the  newest  forums  is  the 
Internet  listserv  "cni-copyright",  a  moderated  discussion  list  on  copyright  and  intellectual  property  rights 
issues.  Educational  issues  are  occasionally  discussed  on  this  list.  It  would  be  desirable,  as  some  of  the 
intervewees  suggested,  to  have  a  listserv  dedicated  to  educational  copyright  issues  with  a  content  expert  to 
moderate  the  discussion.  Perhaps  one  of  the  professional  organizations  could  sponsor  and  moderate  such  a 
list. 

Additional  study  is  needed  to  collaborate  the  findings  of  this  and  previous  studies.  Different  data 
collection  methods  will  help  validate  the  results  of  this  study,  as  well  as,  provide  additional  insight.  One 
suggestion  would  be  to  query  members  in  person,  perhaps  at  a  national  convention.  By  asking  a  group  of 
them  the  same  question  at  the  same  time,  the  results  may  provide  unique  findings. 

Different  populations  could  be  examined.  The  relationship  between  the  MLA  degree  professionals 
and  non-MLA  professionals  provided  little  data.  By  using  a  similar  instrument  with  American  Library 
Association  non-print  media  librarians  and  contrasting  the  results  from  this  study,  a  significant  relationship 
may  be  apparent.  Another  study  might  look  at  the  relationship  of  the  faculty's  knowledge  level  versus  the 

media  director's  knowledge  le.  el .  Does  a  well  informed  director  with  a  clear  copyright  policy  translate  into  a 
faculty  that  has  a  basic  understanding  of  the  law? 

The  revision  of  the  instrument  needs  to  continue  for  any  further  study.  The  questions  should 
continue  to  be  revised.  As  more  guidelines  become  available  for  distance  learning,  multimedia,  and 
computer  software,  they  should  be  included  in  the  instrument.  It  should  also  be  possible  for  the  respondents 
to  indicate  "not  applicable"  as  a  response.  If  an  area  of  the  law  is  not  relevant  to  their  work,  it  should  not  be 
included  in  the  analysis  of  their  performance.  The  response  of  "don't  know"  might  also  be  added  to  the  list 
of  choices  on  the  instrument.  This  would  help  clearly  identify  areas  of  the  law  with  which  the  population  is 
unfamiliar. 

Summary 

The  educating  of  media  directors  across  the  country  needs  to  continue.  If  media  directors  are  to  take 
full  advantage  of  the  educational  exemptions  without  infringing  on  exclusive  rights,  they  must  have  a 
thorough  understanding  of  the  law.  Many  educators  are  frustrated  by  the  limitations  of  the  Copyright  Act. 
but  if  they  fully  understood  the  law  and  its  intent,  they  would  understand  the  liberal  degree  of  latitude  that  is 
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available.  While  most  directors  have  some  knowledge  of  the  law,  only  18%  can  demonstrate  a  criterion 
level  of  competency.  The  area  of  fair  use  is  one  specific  aspect  that  requires  additional  work.  Professional 
associations  like  AECT  and  CCUMC  need  to  continue  their  work  and  disseminate  information  ak;ut 
copyright.  Directors  need  to  also  take  the  initiative  to  own  and  refer  to  copyright  reference  materials.  The 
ethics  of  the  media  profession  are  continually  being  molded  by  the  behaviors  of  media  directors  across  the 
country. 

A  proficient  knowledge  of  the  copyright  law  should  be  a  priority  for  every  media  professional. 
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I.  Research  Background 


Learners  should  be  able  to  feel  free  to  optimize  their  potential  and  to  hold  their  own  in  any  setting. 
Becoming  empowered  means  learning  how  to  influence  and  interact  with  the  challenges  of  one's  life  in 
such  a  way  as  to  maximize  performance.  We  live  in  an  age  of  empowerment.  The  term  empowerment  has 
come  into  use  in  public  education  and  the  business  worlds.  Just  as  the  individual  school  is  viewed  by  most 
people  as  the  basic  primary  source  of  change  and  improvement,  so  too  is  the  training  and  development 
center  in  the  corporate  environment. 

Educators  concerned  about  the  empowerment  of  learners  have  seen  that  a  resi  ucturing  of  schools 
may  be  necessary  if  empowerment  of  learners  is  to  be  realized.  So  it  is  that  the  notion  of  "school 
restructuring"  has  been  discussed  in  the  same  context  as  empowerment(Banathy,  1987, 1991;  Bateman, 
1990;  Elmore,  1989;  Harvey  &  Crandall,  1988;  Murphy,  1991).  In  other  words,  learners  should  be  at  the 
heart  of  educational  processes.  Meanwhile,  educators  should  facilitate  and  empower  learners  to  take  control 
of  their  own  learning(Chung,  1991). 

Empowerment  would  necessarily  involve  a  consideration  of  learner  control  or  self-instructional 
management.  Learner  control  has  been  one  of  the  most  important  issues  in  t"    field  of  instructional 
technology  and  especially  it  is  the  major  instructional  management  strategy  (Heinich,  1973;  Hoban,  .'965; 
Reigeluth,1989).  Each  learner's  control  of  instruction  is  inherently  appealing  to  learners,  since  it  is 
assumed  that  learners  will  be  more  motivated  if  they  are  allowed  some  control  over  their  own  learning. 

Research  findings  regarding  the  effects  of  learner  control  have  been  inconclusive  (Carrier, 
Davidson,  &  Kalweit,  1986;  Ross  &  Morrison,  1989;  Steinberg,  1977;  Tennyson,  1980).  Whatever  the 
causes  for  the  inconclusiveness  of  the  research  into  the  effects,  it  is  time  for  us  to  count  on  the  prescriptive 
knowledge  base  of  learner  control  as  a  means  to  empower  learners.  Educators  must  require  more  systematic 
guidelines  for  implementing  effective  learner  control  in  their  instructional  situations  in  order  to  maximize 
the  learners'  performance. 

The  purpose  of  this  study  is  to  develop  a  prescriptive  theory  of  learner  control  for  educators  to 
support  the  learner's  decision  making  in  the  learning  process  and  to  manage  more  efficiently  their 
instructional  process.  The  theoretical  framework  of  learner  control  in  conjunction  with  the  self-regulation 
of  learning,  learner  characteristics,  and  learner  motivation  may  make  educators  and  instructional  designers 
understand  why  it  may  be  effective  to  allow  learners  to  have  some  control  over  the  learning  process.  In  this 
study,  the  author  assume  that  development  of  an  instructional  theory  for  learner  control  would  answer  some 
needs  of  our  information  society.  This  theory  would  facilitate  educators'  new  roles  by  providing  practical 
guidelines  for  more  effective  instructional  management.  Thus,  educators  can  help  their  students  to  take 
control  of  their  own  learning  process  with  confidence  and  responsibility.  In  a  nutshell,  the  author  expei!^ 
this  prescriptive  theory  to  be  a  meaningful  mechanism  to  empower  learners  in  our  society. 


II.  Research  Questions 

The  major  questions  of  this  study  were: 

(1)  What  are  the  important  factors  required  to  empower  learners  in  managing  their  learning?  What  is 
the  functional  relationship  among  those  factors? 

(2)  What  types  of  learner  control  or  learners'  decision-making  roles  are  available  in  learning  situations? 
What  would  be  the  educator's  role  to  enhance  and  support  the  learner's  own  learning  process  before, 
duri  ig,  and  after  instruction? 

(3)  Under  what  conditions  (condition  and  outcome  variables)  can  a  specific  learner-control  strategy  be 
considered  as  an  associated  solution  or  a  combination  of  solutions  (method  variable)?  In  other 
words,  when  is  and  isn't  a  certain  learner-control  strategy  (or  a  combination  of  strategies) 
recommended''  What  are  the  associated  critical  success  factors'' 

Research  questions  1  and  2  will  be  mainly  answered  by  review  of  relevant  literature.  Question  3 
dial  pail  ill  question  ?  will  be  clarified  and  answered  through  the  instructional  theory  construction  process 


using  inductive  and  deductive  approaches  based  on  empirical  research,  theoretical  research,  and  intuition.  As 
a  matter  of  fact,  the  work  of  theory  construction  is  iterative  and  recycling.  After  all,  the  questions  (1,2, 
and  3)  described  above  can  be  answered  altogether  and  interrelated  in  the  systemic  viewpoint. 

III.  Research  Procedure 

We  need  to  know  the  reason  why  the  author  exploit  the  methodology  (theory  construction, 
especially  using  both  deduction  and  induction  methods)  and  how  to  progress  this  work. 

A.  Rationale 

In  the  current  study,  I  exploit  both  the  inductive  and  the  deductive  approaches.  The  inductive 
method  is  primarily  adopted  in  analyzing  the  empirical  research,  building  a  knowledge  base  of  learner 
control  and  learner  decision-making  skills,  and  constructing  the  current  prescriptive  theory  for  teachers 
which  will  support  learners'  appropriate  control  decisions  during  the  instruction  and  learning  processes. 
According  to  Snelbecker  (1985),  in  the  inductive  mode  of  theory  construction,  statements  are  summarized  or 
generalizations  are  derived  from  empirical  facts.  Such  theories  work  "from  the  bottom  up,"  developing  into 
higher-level  systems  which  generalize  across  small  theories,  and  eventually  culminating  in  a  theory  which 
can  account  for  all  the  statements  lower  in  the  schema.  The  strength  of  this  approach  is  that  the  statements 
produced  are  not  too  far  from  the  "truth"  which  has  been  verified.  The  weakness  of  this  inductive  thinking 
is  that  it  can  often  lead  to  a  proliferation  of  very  low-level  theories,  many  of  which  are  not  unique  and 
contain  considerable  overlap  in  function(Hoover,  1984;  Noble,  1976;  Snelbecker,  1985;  Turner,  1968). 

Meanwhile,  the  deductive  method  is  used  intuitively  both  in  synthesizing  theoretical  research  and 
connecting  a  knowledge  base  to  the  development  of  taxonomies  which  form  the  core  of  a  conceptual  model 
for  learner  control.  The  conceptual  model  for  learner  control  will  be  the  beginning  stage  of  the  current 
theory  construction.  In  the  deductive  approach,  theorists  work  "from  the  top  down,"  building  a  theory  that 
seems  logical  on  an  "a  priori"  basis  and  then  testing  the  correctness  of  this  theory  (Babbie,  !989;  Noble, 
1976;  Snelbecker,  i985;  Turner,  1968).  This  type  of  approach  is  good  in  terms  of  comprehensiveness, 
breadth,  and  a  consistency  among  rules.  However,  the  problem  in  deductive  thinking  is  the  lack  of 
empirical  foundation  and  a  good  deal  of  needless  research  if  the  majority  of  the  original  postulates  prove  to 
be  inconcct.  In  this  study,  the  author  exploited  both  the  inductive  and  the  deductive  approaches.  Figure  1 
explains  how  the  approaches  were  applied. 
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B,  Procedure 


This  procedure  became  the  general  guidelines  for  the  author  to  build  the  theory.  First  of  all,  a  literature 
review  was  conducted.  Next,  a  conceptual  model  of  learner  control  will  be  developed.  Then,  both  empirical 
and  theoretical  research  findings  will  be  integrated  on  the  basis  of  the  conceptual  model.  Lastly,  a 
prescriptive  instructional  theory  will  be  constructed.  However,  in  fact  the  theory  development  process  was 
very  reiterative  and  recycling  in  all  directions. 

1.  Reviewing  Literature 

Literature  is  the  source  for  a  sketch  of  conceptual  models  and  an  instructional  theory.  Reviewing 
literature  is  the  very  first  step  in  the  current  theory  construction.  This  study  reviews  a  variety  of  literature 
(both  empirical  and  theoretical)  relevant  to  decision  making,  human  performance  and  motivation,  learner 
control,  and  theory  construction.  The  literature  was  obtained  from  discussion  and  consultation  with 
professors  and  colleagues,  from  books,  journal  articles,  conference  papers,  Psychlit,  and  ERIC  microfiches 
across  the  areas  of  education,  psychology,  and  business. 

2.  Developing  Descriptive  Conceptual  Models 

Through  the  literature  review  process,  the  conceptual  models  were  developed.  The  conceptual 
models  in  this  study  has  two  shapes:  one  is  a  diagram  of  variables  (Figure  2);  the  other  is  a  diagram  of  the 
process  (Figure  3).  Figure  2  identifies  and  organizes  the  relevant  variables  affecting  a  learner's  decision 
making  and  describes  relationships  among  the  variables.  Figure  3  shows  the  roles  (interaction  or  relation) 
of  learners  and  teachers  in  a  given  learning  environment.  The  models  were  products  of  the  synthesis  of 
intuition,  and  the  related  research  and  knowledge  base  about  learner  control.  They  facilitated  a  theory 
development.  In  other  words,  the  conceptual  models  were  the  basis  of  development  of  formative  hypotheses 
and  the  taxonomy  of  variables  of  instructional  theory. 

3.  Integrating  Research  Findings 

When  a  quantity  of  relevant  research  data  is  gathered  and  analyzed,  the  research  findings  were 
integrated  on  the  basis  of  the  conceptual  models  so  that  logical  conclusions  can  be  drawn.  As  result  of  this 
activity,  a  set  of  principles  were  produced.  Each  principle  consists  of  condition(s)  and  method(s).  The 
credibility  of  a  theory  is  dependent  upon  the  component  constructs,  as  well  as  the  substance  of  the 
principles  of  the  theory. 

4.  Constructing  the  Prescriptive  Instructional  Theory 

Inductively  integrated  knowledge  which  is  based  on  research  findings  concerning  learner  control 
was  changed  into  a  prescriptive  form  in  order  to  provide  teachers  and  instructional  designers  with  a  guide  to 
empowering  learners.  After  identification  of  four  major  sets  of  conditions,  the  current  study  produced  the 
theory  by  prescribing  four  instructional  models  of  learner  control  to  cope  with  individual  learning  and 
instructional  situations.  The  deductive  approach  was  also  used  to  improve  the  framework  of  the  theory. 

On  the  basis  of  Conceptual  Model  l(Figure  2)  with  integrated  research  findings,  critical  condition 
variables  (such  as  learner  ability,  prior  knowledge,  types  of  learning,  task  importance),  critical  method 
variables(such  as  content  control,  sequence  control,  pace  control,  and  display  control),  and  critical  outcome 
variables  (or  dependent  variables  such  as  achievement  and  attitude)  were  identified.  The  interrelationships 
between  the  critical  variables  were  identified.  At  this  point,  prescriptive  principles  were  matched  with 
situations. 

Based  on  method  variables,  four  instructional  models  (A,  B,  C,  D)  were  constructed  to  match  the 
four  conditions.  First,!  decided  that  each  model  should  consist  of  three  instructional  tactics:  (1)  providing 
learners  with  an  introduction,  delivered  by  an  instructor  or  a  pr  'gram,  about  how  to  use  learner  control 
effectively  in  the  given  program,  (2)  allowing  learners  to  use  some  degree  and  type  of  control  options,  and 
(3)  providing  instructhtuil  (nr  control)  advk  e  during  the  program.  Then,  the  second  kind  of  instructional 
tactics  fallowing  learners  u>  use  instiucliixml  options  available  in  the  progress  of  learning)  »<-cre  mainly 
inductively  identified  from  Conceptual  Model  I .  IIovvlvci.  tr  •  first  and  the  third  tactics  were  identified 


primarily  deductively  on  the  basis  of  Conceptual  Model  2.  As  a  result  of  induction,  deduction,  and 
integration  of  conceptual  models,  Figure  4  (the  overall  picture  of  the  theory)  was  created.  All  the  key 
principles  under  the  critical  success  factors  (CSF)  in  each  model  were  listed. 


IV.  Results 

The  results  of  this  study  are  composed  of  two  descriptive  conceptual  models  and  a  prescriptive  instructional 
theory  for  learner  control  in  which  includes  four  instructional  models  . 

A.  Descriptive  Conceptual  Models 

1.  Conceptual  Model  l(Variable  Relationship) 

There  are  three  groups  of  variables:  condition,  method,  and  outcome  variables.  Condition  variables 
include  learner  variables  (such  as  age,  aoility,  prior  knowledge,  cognitive  style,  and  motivational  level), 
content  variables  (such  as  learning  type  and  task  importance),  and  environment  variables  (such  as  climate 
and  medium).  Method  variables  (i.e.  learner  control  strategies  in  present  study)  include  content  control, 
sequence  control,  pace  control  (time  management),  and  display  control.  Each  method  variables  will  be 
defined  later  in  the  prescriptive  instructional  theory  learner  control.  Outcome  variables  include  learner 
performance  (or  achievement),  learner  attitude,  continuing  motivation  to  learn,  number  of  selection, 
anxiety,  and  learning  time. 
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Figure  2.  Conceptual  Modal  1  (Vtrltbla  Ralaiionahlpa) 
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As  a  result  of  the  integration  of  empirical  and  theoretical  findings  on  the  conceptual  model  1,  the 
author  could  identify  the  relationships  among  the  variables  as  Figure  2.  These  relationships  are  the  basi?  of 
the  prescriptive  instructional  theory  of  learner  control.  Figure  2  indicates,  on  the  basis  of  the  number  of 
variable  relationship  involved,  that  achievement  and  attitude  to  a  large  extent,  and  aaxiety  and  learning  to  a 
less  extent,  are  decisive  dependent  variables.  Especially,  the  degree  of  learner  experience  (ability  and  prior 
knowledge),  the  degree  of  task  importance,  and  types  of  learning  are  the  critical  factors  of  condition 
variables  in  the  present  theory. 

2.  Conceptual  Model  2(Self-Managed  Learning  Process) 

Figure  3  describes  the  self-managed  learning  process.  A  learner  (1.0)  becomes  an  empowered 
learner  (3.0)  through  the  learning  process  (2.0).  The  learning  process  consists  of  a  session  of  training  about 
decision  making  (learning  strategies),  self-managed  learning  experience,  and  self-assessment  by  the  learner. 
In  conceptual  model  2  (Figure  3),  we  can  identify  the  roles  of  a  learner  and  an  educator.  The  learner  will  be 
the  center  of  the  learning  process  and  he/she  will  be  an  active  participant  in  the  whole  learning  process  with 
self-regulation  skills.  The  educator  can  provide  knowledge  and  skills  in  the  pre-training  session.  The 
educator  can  also  advise,  counsel,  facilitate,  and  influence  learners  during  the  learners'  self-managed  learning 
experiences.  The  educator  can  provide  learners  with  informative  feedback  during  or  after  learning  session. 
During  this  process,  a  learner's  schemata  (cognitive  structure)  changes  one  time,  either  by  a  process  of 
accretion,  or  a  process  of  tuning,  or  a  process  of  restructuring.  Often  a  meta-process  is  involved  which 
consists  of  a  mixture  of  all  three  processes,  with  the  mixture  varying  one  time. 
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B.  A  Prescriptive  Instructional  Theory 

This  theory  prescribes  what  models  based  on  different  conditions  are  likely  to  optimize  the  desired 
instructional  outcomes.  Four  distinct  models  operationalize  this  instructional  theory  for  learner  control. 
As  in  Figure  4,  Model  A  is  most  learner-control  oriented  and  Model  D  is  least  learner-control  oriented  (a 
continuum). 

1.  Taxonomy  of  Variables  for  the  Instructional  Theory 

Instructional  GoaliOutcome):  For  all  four  models  in  this  theory,  the  desired  learning  outcome  (that  is, 
instructional  goal)  is  to  meet  different  individual  leiirner  needs.  Instruction  is  judged  by  its  effectiveness, 
efficiency,  and  appeal  in  the  context  of  instructional  management  (Davies,  1984;  Reigeluth  &  Merrill, 
1979). 

Instructional  Conditions:  The  selection  of  instructional  methods  in  this  theory  is  mostly  determined  by  two 
sets  of  condition  variables:  1)  learner  experience  and  2)  task  importance  (Korofkin,  1992;  Davies,  1993; 
Grau,  1986;  McMillan  &  Spratt,  1983;  Weiner  &  Brown,  1984  ). 

Condition  1 :       If  the  learner  has  a  high  experience  and 

if  the  task  is  less  crucial; 
Condition  2:       If  the  learner  has  a  low  experience  and 

if  the  task  is  less  crucial; 
Condition  3:       If  the  learner  has  a  high  experience  and 

if  the  task  is  more  crucial; 
Condition  4:       If  the  learner  has  a  low  experience  and 

if  the  task  is  more  crucial. 


Instructional  Methods:  In  this  section,  instructional  methods  are  divided  into  the  types  of  learner  control 
exploited  in  the  field,  the  role  of  educators,  and  the  degree  of  learner  control. 

a)  Tvpt  s  of  Learner  Control:  Roles  of  active  learners 

These  learning  activities  can  be  arranged  into  four  broad  classes  of  control:  learner  control  of  content,  learner 
control  of  sequence,  learner  control  of  pace,  and  learner  control  of  instructional  display  (strategy). 

-  Learner  control  of  content.  (Merrill,  1983). 

-  Learner  control  of  sequence.  (Merrill,  1979;  Milheim  &  Martin,  1991;  Reigeluth  &  Curtis,  1987). 

-  Learner  control  of  pace  (time  management).  (Milheim  &  Martin,  1991). 

-  Learner  control  of  instructional  display  (strategy).(MemW,  1984;  Ross,  Morrison,  &  O'Dell,  1989;  Ross 
&  Morrison,  1989). 

b)  Role  of  Educators 

-  Developing  metacognition  and  cognitive  strategies  of  learners.  (Merrill,  1984;  Flavell,  1976;  Resnick, 
1972;  Corno  &  Mandinach,  1983;  Gagne"  &  Glaser,  1987). 

-  Instructional  advisement.  (Milheim  &  Azbell,  1988;  Tennyson  &  Rothen,  1979). 

c)  Degree  of  Control:  The  degree  of  control  is  a  continuum  from  maximum  program  control  (that  is, 
minimum  learner  control)  to  maximum  learner  control. 

2.  Four  Instructional  Models  of  Learner  Control. 

Four  prescriptive  instructional  models  (Model  A,  B,  C,  and  D)  will  be  elaborated  here.  Each  will 
be  described  in  terms  of  the  tactics  involved  and  each  will  identify  a  precept  or  rule  as  a  critical  success 
factor  (CSF)  to  the  model. 
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Model  A 

If  the  learner  has  a  high  experience  (ability  and  prior  knowledge)  and  the  given  task  is  not  crucial 
("Condition  1")  in  the  learning,  then  the  optimal  prescription  for  instructors  or  instructional  designers  in 
this  instructional  theory  for  learner  control  is  Model  A. 


(Goal:  To  meet  individual  learner  needs) 
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Model  B 

*  Introduction 

*  Control 

•  Content  Selection:  Learner  control 

•  Sequence  (Path):  Learner  control 

•  Time  Management  Learner  control 

•  Display  (Strategy):  Program  control 

*  Instructional  Advice 
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Model  C 

*  Introduction 
•Control 

•  Content  Selection:  Program  control 

•  Sequence  (Path):  Program  control 

•  Time  Management:  Learner  control 

•  Display  (Strategy):  Learner  control 

*  Instructional  Advice 
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Model  D 

*  Introduction 

*  Control 

•  Content  Selection:  Maximum  program  control 

•  Sequence  (Path):  Maximum  program  control 

•  Time  Management:  Learner  control 

•  Display  (Strategy):  Program  control 

*  Instructional  Advice 


Figure  4 

Models  of  Instructional  Theory  for  Learner  Control 


The  three  instructional  tactics  in  this  model  are:  a)  providing  the  introduction  ortmini.ig  on  learner 
control;  b)  allowing  learners  to  use  instructional  options  available  in  the  progress  of  learning;  and  c) 
providing  instructional  advisement  for  effective  learning.  Usually  c)  is  embedded  in  b).  In  condition  1 , 
tactic  a)  is  less  important  than  in  other  conditions. 
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a)  Critical  success  factors  for  learning  strategies  training: 

CSF:     Provide  learners  with  directions  in  how  to  use  learner  control  strategies  in  order  to  promote  a 
learner's  performance. 

CS£:     Provide  learners  with  directions  for  conscious  cognitive  processing  of  the  information  in  order  to 
improve  their  performance  (Callahan  &  Merrill,  1978;  Merrill,  1984;  Reigeluth, 1979;  Strickland, 
Fletcher,  &  Merrill,  1978;  Wilcox,  Richards,  Hindmarsh,  &  Merrill,  1978). 

b)  Critical  success  factors  for  experiencing  control  options: 

Cg£:     A  high  degree  of  learner  control  over  content  selection  (what 

to  learn )  is  advisable  when  it  is  beneficial  for  learners  to  set 

their  own  learning  goals(Ross  &  Rakow,  1981;  Steinberg,  1991). 
CSF:     High  degree  of  learner  control  over  sequence  of  content  is  advi'saMe.  (Leshin,  Pollock,  & 

Reigeluth,  1992;  Reigeluth,  1979) 
CSF:     A  high  degree  of  learner  control  over  instructional  pacing  should  be  a//ow£*/.(Keller  &  Kopp, 

1987;  Leshin,  Pollock,  &  Reigeluth,  1992;  Milheim  &  Martin,  1991). 
CSF:     A  high  degree  of  learner  control  over  strategy  should  be  allowed(Beard,  Lorton,  Searle,  & 

Atkinson,  1973;  Leshin,  Pollock,  &  Reigeluth,  1992;  Milheim  &  Martin,  1991).  Men  ill,  1983; 

Merrill,  1984;  Reigeluth  &  Merrill,  1977;  Reigeluth  &  Schwarts,  1989;  Slough,  Ellis,  &  Lahey, 

1972). 

c)  Critical  success  factor  for  instructional  advisement 

CSF:     Provide  learners  with  instructional  advisement  only  when  necessary  in  order  to  facilitate  at  d 

improve  learner's  decision  making  and  performance  (Hannafin,1984;  Kinzie,  1990;  Laurillard,1984; 
Merrill,  Li,  &  Jones,  1990:  Tennyson  &  Park,  1987). 


Model  B 

If  the  learner  is  less  experienced  and  the  given  task  is  less  crucial  ("Condition  2")  in  the  learning, 
then  the  optimal  prescription  for  i nstructors  or  instmctional  designers  in  this  instructional  theory  for  learner 
control  is  Model  B.  Like  Model  A,  there  are  also  three  instructional  tactics  in  this  model: 

a)  Critical  success  factors  for  learning  strategies  training: 

qsF:     Absolutly  provide  learners  with  directions  in  how  to  use  learner  control  strategies  in  order  to 

promote  a  learner's  performance. 
CSF:     Absolutely  provide  learners  with  directions  for  conscious  cognitive  processing  of  the  information 

n  order  to  improve  their  performance. 

b)  Critical  success  factors  for  experiencing  control  options: 

CSF:  Learner  control  over  content  selection  ( what  to  learn)  is  advisable. 

CSF:  Learner  control  over  sequence  of  content  is  advisable. 

CSF:  High  degree  of  learner  control  over  instructional  pacing  should  be  allowed. 

CSF:  Learner  control  over  strategy  should  be  restricted. 

c)  Critical  success  factor  for  instructional  advisement 

CSF:     Provide  learners  with  instructional  advisement  only  when  necessary  in  order  to  facilitate  and 
improve  learner's  decision  making  and  performance. 


If  the  learner  has  a  high  experience  and  the  given  task  is  more  crucial  ("Condition  3")  in  the 
learning,  then  the  optimal  prescription  for  instructors  or  instructional  designers  in  this  instn-ctional  theory 
for  learner  control  is  Model  C.  There  are  also  three  instructional  tactics  in  this  model: 

a)  Critical  success  factors  for  learning  strategies  training: 

CSF:     Absolutely  provide  learners  with  directions  in  how  to  use  leumercontrol strategies  in  order  to 
promote  a  learner's  performance. 


*  Si 

ERIC  80 

ummmmmM 


CSF:     Absolutely  provide  learners  with  directions  for  conscious  cognitive  processing  of  the  information 
in  order  to  improve  their  performance. 


b)  Critical  success  factors  for  experiencing  control  options: 


£££ 


CSJE 


CSF 


CSJE 


Learner  control  over  content  selection  ( what  to  learn)  is  restricted. 
Learner  control  over  sequence  of  content  is  restricted. 
High  degree  of  learner  control  over  instructional  pacing  should  be  allowed. 
High  degree  of  earner  control  over  strategy  should  be  allowed. 


c)  Critical  success  factor  for  instructional  advisement 

CSF:     Provide  learners  with  instructional  advisement  only  when  necessary  in  order  to  facilitate  and 
improve  learner's  decision  making  and  performance. 

Model  D 

If  the  learner  has  a  low  experience  and  the  given  task  is  very  crucial  ("Condition  4")  in  the 
learning,  then  the  optimal  prescription  for  instructors  or  instructional  designers  in  this  instructional  theory 
for  learner  control  is  Model  D.  There  are  three  instructional  tactics  in  this  model: 

a)  Critical  success  factors  for  learning  strategies  training: 

CSF:     Provide  learners  with  directions  in  how  to  use  learner  control  strategies  if  necessary. 

CSF:     Provide  learners  with  directions  for  conscious  cognitive  processing  of  the  information  if  necessary. 

b)  Critical  success  factors  for  experiencing  control  options: 


CSF 
QSE 
CSE: 
CSF: 


Learner  control  over  content  selection  is  restricted. 
Learner  control  over  sequence  of  content  is  restricted. 
High  degree  of  learner  control  over  instructional  pacing  should  be  allowed. 
Learner  control  over  strategy  should  be  restricted. 


c)  Critical  success  factor  for  instructional  advisement 

CSF:     Provide  learners  with  instructional  advisement  only  when  necessary  in  order  to  facilitate  and 
improve  learner's  decision  making  and  performance. 
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Figure  5  is  a  summary  table  of"  learner  control  strategies  of  each  model.  All  learner  control  strategies  are 
allowed  in  Model  A.  Meanwhile,  most  learner  control  strategies  are  restricted  in  Model  D. 


more  LC 


less  LC   ^ 
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Model  A 

Model  B 
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Model  D 

Introdi 
(Pre-tr 

iction 
aining) 

less 

important 
for  some 

important 

important 

important 

Control 
Options 

Content 
Control 

maximum 

learner 

control 
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control 
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(max.  PC) 

Sequence 
Control 
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control 
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control 
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LC 
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minimum 
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(max.PC) 

Pace 
Control 

learner 
control 

learner 
control 
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control 

Display 
Control 

learner 
control 

restricted 
LC 
(PC) 

learner 
control 
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Instructional 
Advice 

important 

important 

important 

important 

•  LC:  learner  control 

•  PC:  program  control 
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V.  Conclusions  and  Educational  Implications 


The  results  of  this  study  permit  several  conclusions  about  the  instructional  theory  construction  for 
learner  control.  The  following  conclusions  seem  noteworthy. 

1 .  There  are  three  major  variables  affecting  learning  process  and    learning  control  decisions.  Those 
are  condition  variables,     method  variables,  and  outcome  variables. 

2.  The  most  important  critical  variables  which  influence  the  learner  control  decision  (method 
variables)  are  (a)  the  experience  of  a  learner  and  (b)  the  importance  of  task.  The  combination  of 
these  two  variables  creates  four  instructional  conditions  of  my  learner  control  theory. 

3.  A  learner's  role  in  the  self-management  learning  environment 

is  to  be  an  active  participator  with  self-regulation  skills  (metacognition). 

4.  The  role  of  an  educator  or  instructional  agent  in  the  self-management    learning  environment  is  to 
be  a  facilitator  and  mediator  of  learning  with  instructional  and  organizational  roles. 

5.  There  are  four  instructional  conditions  of  learner  control  theory  and  four  instructional  models  for 
matching  each  instructional  situation.  Each  instructional  model  has  three  instructional  tactics: 

(a)  Developing  self-regulation  skills  of  a  learner. 

(b)  Allowing  a  learner  to  use  the  degree  and  the  type  of  control. 

(c)  Providing  instructional  advice  during  learning. 

6.  There  are  several  CSFs  which  are  critical  to  the  success  of  the  instructional  theory  prescriptions  for 
learner  control. 

This  theory  expands  previous  conceptualizations  of  the  roles  of  learners  and  teachers  by  exploring 
the  decision-making  process  and  the  interrelationships  of  learner  factors,  content  factors,  environment 
factors,  and  delivery  factors.  This  theory  links  previous  research  findings  and  practitioners'  prescriptions 
within  a  logically  consistent  framework.  Theoretical  research,  because  it  deals  with  abstraction  and 
relationships,  is  often  more  demanding  than  other  forms  of  inquiry.  However,  theory  construction  is 
essential  to  support  or  question  the  foundations  of  practice. 

This  study  takes  a  first  documented  step  toward  theory  construction  for  empowering  learners  to 
manage  the  learning  process.  It  is  a  exploratory  attempt  to  improve  the  common  practice  of  education  i.nd 
training.  The  recommendations  resulting  from  this  study  fall  into  two  main  categories:  one  is  for 
evaluation  of  the  theory  developed  in  this  study;  the  other  is  for  expansion  of  the  theory. 

For  evaluation  of  the  theory: 

1 .  In  the  future,  this  theory  must  be  examined  within  the  limits  of  experimental  research  desigr .  The 
investigation  results  can  serve  as    further  tests  of  the  components  of  the  theory. 

2.  The  theory  can  also  be  evaluated  by  a  large  sample  of  experts  and  practitioners  (teachers  and 
trainers).  It  is  expected  that  further  research  and  testing  efforts  will  lead  to  refinements. 

The  use  of  experts'  opinions  for  assessing  the  worth  of  a  product  is  probably  the  oldest  evaluation 
strategy  used  in  education.  Experts'  and  practitioners'  opinions  can  be  an  important  evaluation 
tool  because  it  is  quick  and  enhance  the  credibility  of  the  theory. 

The  instructional  prescriptions  developed  in  this  study  were  validated  with  empirical  and  theoretical 
research  by  the  author.  But  the  instructional  theory  need  also  to  be  validated  in  terms  of 
optimality  and  utility  by  both  experts  in  the  fi  Id  of  instructional  theory  and  learner-control 
research  and  practitioners  in  the  education  and  training.. 

For  expansion  of  the  theory: 

I .  Additional  work  can  be  done  to  find  the  relationship  between  motivational  design  and  learner 
performance  in  the  self-managed  learning  environment.  The  motivational  effects  is  expected  to 
make  the  instructional  theory  for  learner  control  more  dynamic. 
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2. 


Additional  work  should  be  done  to  develop  more  flexible  instructional  theory  for  learner  control  in 
order  to  cope  with  advanced  information  technologies  including  intelligent  tutorial  systems,  expert 
systems,  and  computer-based  simulations  and  games. 
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OBJECTIVE 

The  task  of  synthesizing  descriptive  data  from  the  "root  social  sciences"  (Clark,  1989)  to  form  a  usable 
model  for  applied  research  is  quite  different  than  the  task  of  employing,  verbatim,  the  theoretical  and 
methodological  constructs  of  those  same  sciences.  Certainly  learner  competence  is  a  goal  or  end  product 
that  applied  fields  concerned  with  instructional  theory  hold  in  common  with  the  root  social  sciences 
concerned  with  learning/cognitive  theory,  but  the  means  of  arriving  at  the  end  product  is,  or  should  be, 
different  Therefore,  the  objective,  was  to  conceptualize  (in  the  phonemic  tradition)  the  instructional 
encoding  conditions  that  potentially  effect  and/or  affect  the  encounters  of  predisposed  learners/decoders 
involved  in  perceiving,  differentiating,  recalling,  manipulating,  and/or  using  knowledge. 

THEORETICAL  FRAMEWORK 

Perhaps  the  most  pervasive  problem  with  research  on  learning  from  instruction  has  been  the  lack  of 
inclusiveness-  our  inattention  to  the  totality  of  the  instructional  environment.  Separately,  we  have  covered 
several  variables,  usually  more  than  one  at  a  time,  including  learner,  teacher,  and  treatment  characteristics, 
environmental  or  situational  conditions,  intrinsic  and  extrinsic  motivation,  media  characteristics,  response 
characteristics,  and  factors  related  to  the  instructional  message  or  task. 

In  the  past,  our  commitment  to  the  constructs  imposed  by  a  particular  discipline  has  i'r;Lid  to  produce  the 
consistency  generality,  and  commonality  needed  for  the  development  of  a  comprehensive  model  tor  applied 
research  on 'learning  from  instruction.  In  the  future,  the  relations  between  learning/cognitive  theory  research 
and  learning  from  instruction  research  must  be  reciprocal:  each  should  serve,  in  part,  to  strengthen  the 
other. 

Without  some  acknowledgment  of  the  effective  and  affective  processes  involved  in  learning  from 
instruction,  applied  research  efforts  will  continue  to  produce  extremely  limited  observations.  Learning  from 
instruction  should  be  based,  in  part,  upon  the  constructs  of  learning/cognitive  theory,  but  instructional 
theory  should  not  be  confused  with  learning/cognitive  theory. 

TECHNIQUE 

The  path  model  (Figure  1)  depicting  the  ontological  dependence  and  functional  relations  (see  Fenstermacher, 
1986  and  Travers  1981  for  a  discussion  of,  and  rationale  for,  these  terms)  of  the  variables  of  instruction 
was  patterned  after  the  LISREL  7  model  by  Joreskog  &  Sorbom  (1989).  Inferencing  strategies  were 
employed  to  identify  qualitative  and  theoretical  surface  features  that  may  or  may  not  be  considered  during  the 
encoding  phase  of  instruction,  but  that  are  certainly  potential  indicators  of  interactions  during  the  decoding 
phase  of  instruction.  The  model  is  nonrecursive,  with  both  temporal  and  conceptual  sequences  employed  to 
estimate  pathways  of  influence.  The  model  is  based  upon  the  presumption  that  there  is  both  theoretical  and 
intuitive  commonality  among  researchers  and  practitioners  interested  in  learning  from  instruction^  For  such 
a  model  to  be  of  value  to  all  researchers  within  an  applied  field  of  study,  it  must  be  eclectic.  It  must  be 
broad  enough  to  include  our  quantitative  and  qualitative  research  interests,  and  dynamic  (nonlinear)  enough 
to  allow  for  the  inclusion  of  various  theories  pertaining  to  what,  where,  when,  how,  which,  and  why 
learners  learn.  The  model  was  used  as  a  heuristic  technique  (not  as  an  analytical  device)  to  conceptualize, 
albeit  naive,  a  priori  encoding  conditions. 

NOTATION  AND  DEFINITIONS 

Several  types  of  variables  are  illustrated  in  Figure  1.  The  Xj's  represent  endogenous  variables  which  are 
effected/affected  by  other  variables  from  within  the  model.  The  Yfs  (innate  qualities)  and  the  Zj 
(experiential  qualities)  are  termed  exogenous  variables  since  they  are  not  effected/affected  by  other  variables 
from  within  the  model  during  the  time  pre-instructional  encoding  conditions  are  being  considered 
Exogenous  variables  are,  in  fact,  the  inherent  substance  of  the  endogenous  variables:  the  differentiated 
developmental  modes  innately  existing  within,  acquired  by/from  experience  (socioculturally  constructed),  or 
designed  into  the  variables  cf  instruction.  The  double-headed  curved  arrows  represent  unanalyzed 
correlations  betweer  exogenous  variables.  The  Rj's  represent  residual  variables  (error  variances)  that 
impinge  upon  the  other  variables  within  the  model,  but  for  which  no  observations  are  gathered.  Two  kinds 
of  intermediary  relations  are  depicted  in  the  model.   Functional  relations  between  the  variables  are 
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represented  by  solid  arrows;  however,  no  causal  relations,  in  the  Newtonian  sense,  were  intended. 
Similarly,  ontologically  dependent  relations  are  represented  by  dashed  arrows.  The  Py's  are  referred  to  as 
path  coefficients  that  represent  the  estimated  impact  of  one  variable  upon  another.  (Note  that  only  solid 
arrows  have  such  designations  and  that  for  clarity  some  have  been  labeled  outside  the  model,  above  and 
below  the  respective  solid  arrows.) 

Figure  1.  A  Path  Model  of  Encoding  Conditions  that  Prefigure  Learning  from 


Task  Requirements      X2      Prior  and/or  new  implicit  and/or  explicit  characteristics  of  knowledge 

structures  and/or  processing  demands. 

Resource  Attributes      X3      Qualitative  and/or  theoretical  surface  features  employed  to  present 

and/or  represent  and  convey  temporal,  spatial,  or  spatiotemporal 
characteristics. 

Teacher  Considerations      X4      Predisposed  conceptions  of  prior  and/or  new  implicit  and/or  explicit 

structural  and/or  processing  demands  of  the  task  and  of  the  predisposed 
learner. 

Teacher  Considerations      X5      Predisposed  perceptions  of  qualitative  and/or  theoretical  surface  features 

essential  for  economical  representation  of  the  task  for  the  predisposed 
learner. 

Teacher  Considerations      X0      Predisposed  formulations  of  concomitant  encoding  conditions 

predicated  to  be  indispensable  for  engaging  the  predisposed  learner  with 
the  task. 


Instruction. 


Learner  Processes 


Xl 


Predisposed  modalities  of  inference  (intellectual,  physical,  emotional, 
sociocultural)  that  facilitate  or  suppress  learning  (achievement, 
process)  from  instruction. 
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Intellectual  Modalities      Y7      Differentiated  developmental  modes  innately  existing  within,  acquired 

by/from  experience  (sociocuiturally  constructed),  or  designed  into  the 
variables  of  instruction. 

Physical  Modalities      Ys      Differentiated  developmental  modes  innately  existing  within,  acquired 

by/from  experience  (sociocuiturally  constructed),  or  designed  into  the 
variables  of  instruction. 

Emotional  Modalities      Y9      Differentiated  developmental  modes  innately  existing  within,  acquired 

by/from  experience  (sociocuiturally  constructed),  or  designed  into  the 
variables  of  instruction. 

Sociocultural  Modalities      Z\Q     Differentiated  developmental  modes  acquired  by/from  experience 

(sociocuiturally  constructed)  or  designed  into  the  variables  of 
instruction. 


POINT  OF  VIEW 

The  encoding  conditions  that  prefigure  learning  from  instruction  consist  of  an  instructional  environment 
tha'  is  composed  of  four  variables:  the  teacher,  the  learner,  the  task,  and  the  resources.  It  follows,  then, 
that  if  learning  (achievement,  process)  is  the  ultimate  product  of  the  instructional  environment,  then  an 
effective  (natural  science)  and  affective  (social  science)  instructional  environment  is  the  product  of  the 
interactions  from  within,  between,  and  among  the  teacher,  the  learner,  the  task,  and  the  resources  (Clark, 
1990).  Intellectual,  physical,  emotional,  and  sociocultural  modalities  have  been  identified  to  further 
describe  the  distinctiveness  of  the  differentiated  developmental  modes  innately  existing  within,  acquired 
by/from  experience  (sociocuiturally  constructed),  or  designed  into  the  variables  of  instruction.  The  model 
not  only  depicts  the  intermediary  relations  between  the  independent  variables  and  the  dependent  variable  of 
interest  (the  predisposed  teacher's  formulation  of  encoding  conditions  predicated  to  be  indispensable  for 
engaging  the  predisposed  learner  with  the  task),  but  also  makes  explicit  the  kinds  of  intermediary  relations 
(functional,  ontological)  that  exist  between  the  variables. 

It  is  noteworthy  to  observe  that  the  functional  relations  (solid  arrows)  specified  in  the  model  represent 
potential  interactions  of  inartificial  innate  qualities  originating  in,  or  derived  from,  the  constitutions  of  the 
teacher  and  of  the  learner.  All  other  intermediary  relations  appearing  in  the  model  represent  ontologically 
dependent  interactions  (dashed  arrows)  of  sociocuiturally  constructed  experiences.  Each  intermediary  relation 
implicitly  represents  an  hypothesis. 

The  functional  relations,  those  solid  arrows  with  path  coefficients  assigned,  could  be  tested  by  estimating 
the  magnitude  of  the  relation.  Since  certain  assumptions  may  not  be  met  and/or  quantitative  data  may  not 
be  available,  equation  systems  for  estimating  the  magnitude  of  the  ontologically  dependent  interactions, 
those  dashed  arrows  without  path  coefficients  assigned,  are  unidentified  at  this  time.  Nevertheless,  while 
quantitative  estimates  of  the  impact  of  one  variable  upon  another  may  not  be  possible  in  every  situation, 
this  heuristic  technique  makes  the  implicit  explicit  by  facilitating  clearer  thinking,  which  may  result  in  the 
generation  of  additional  insights  regarding  research  on  learning  from  instruction.  The  relations  between  and 
among  the  variables  of  instruction  are  never  indifferent  in  their  effect  and/or  affect  upon  learning  from 
instruction.  Therefore,  they  are  of  great  import  to  our  efforts  in  diagnosing  and  prescribing  the  between  and 
among  relations  of  the  variables  of  instruction. 

The  implication  of  such  within,  between,  and  among  dependence  of  the  variables  of  instruction  is  that 
functional  and  ontological  relations  within  the  instructional  environment  result  from  the  abilities  of  the 
teacher  and  learner  to  integrate  the  other  variables  in  a  schema  that  will  result  in  a  theoretically  predictable 
performance.  The  yet-to-be-researched  interactions  that  exist  within  the  path  model  of  encoding  conditions 
may  impinge  upon  current  assumptions  held  by  confirmationists  regarding  uncontrolled  pre-existing 
differences  within,  between,  and  among  the  variables  of  instruction. 
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Research  proceeding  from  this  conceptual  level  requires  not  only  the  interpretation  of  the  variables  of 
instruction  (teacher,  learner,  task,  resources)  and  the  modalities  of  inference  (intellectual,  physical, 
emotional,  sociocultural),  but  of  the  intermediary  relations  (functional,  ontological)  as  well.  Together,  the 
variables  of  instruction,  modalities  of  inference,  and  intermediary  relations  can  serve  as  common 
denominators  for  a  variety  of  research  interests  in  applied  fields  of  study.  Moreover,  theoretical  integrity 
would  be  established  and/or  advanced  if  these  components  were  either  implicit  or  explicit  features  of  most, 
if  not  all,  research  models.  To  view  the  study  of  learning  simply  as  a  natural  science,  to  be  measured  by 
statistics,  is  simplistic,  misleading,  and  false;  learning  is  a  state  of  mind,  a  matter  of  horizons. 
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ABSTRACT 


An  entire  class  of  (37)  of  in-service  government  schools  teachers  at  Nablus  district  was  taken  to 
investigate  the  effect  of  18  hours  of  training  in  Instructional  Designer's  Competencies  (IDC)  on  teachers' 
planning  routine,  and  their  students'  academic  achievement, 

A  questionnaire  measures  (IDC)  in  five  domains:  analysis,  design,  implementation,  management  and 
evaluation,  administered  to  the  teachers  twice:  once  before  the  training,  and  once  after.  At  the  same  time, 
teachers  were  asked  to  teach  two  lessons  of  the  subject-matter  they  teach  in  their  schools  in  one  week,  and  to 
test  their  students  in  the  following  week  immediately.  This  process  of  teaching  and  testing  was  also 
administered  twice:  once  before  training  and  once  after. 

t-test  for  dependent  samples  showed  that  the  overall  mean  of  teachers'  planning  routine  was  enhanced 
significantly  after  receiving  the  training  (t(29),=  -2,86,  p<.0O7}. 

Lindquist  Type  1  ANOVA  and  the  "Scheffe"  test  showed  the 
following  results: 

1-  The  general  (F)  showed  a  significant  main  effect  for  IDC's  domains  { F(4: 1 45)=3. 1 5,  p>.01 },  but  the  post- 

hoc  ANOVA  by  using  the  Scheffe  test  did  not  show  such  significance  among  domains. 

2-  Teachers  practiced  significantly  (p<.05)  the  annual  type  of  planning  better  than  the  monthly  and  seasonally 

ones,  but  there  was  no  differences  between  the  annual  and  daily  types. 

3-  Teachers  practiced  significantly  (p<.05)  the  written  form  of  planning  better  than  practicing  both  forms: 

written  and  mental  together,  but  there  was  no  differences  between  the  written  and  mental  types. 
A-  Only  the  mental  form  of  planning  was  enhanced  significantly  from  before  training  to  after. 

5-  There  was  no  significant  main  effect  between  subjects,  considering  each  of  these  independent  variables: 

gender,  nurr  ber  of  years  of  teaching  experience,  number  of  training  programs  that  the  teachers  have 
taken  during  in-service,  teachers'  college  specialization,  and  the  level  of  learning  stage  that  the  teachers 
teach  in. 

6-  There  was  a  significant  main  effect  at  (.01)  level  of  significance  for  within  subjects,  the  repeated  measure,  for 

each  of  the  above  mentioned  independent  variables  which  indicated  that  teachers'  planning  routine  was 
enhanced  significantly  after  training. 

7-  There  was  no  significant  interaction  between  the  repeated  measure  and  each  of  the  above  mentioned  variables 

except  for  the  learning  stage  (p<  0.007).  This  interaction  indicated  that  the  planning  routine  of 
elementary  school  teachers  was  increased  after  training.  Whereas,  the  planning  routine  of  teachers  who 
teach  all  grades  (1-12)  was  decreased,  and  vice  versa  before  the  training. 

The  "Pears on  correlation  coefficient"  between  teachers'  planning  routine  and  their  students'  academic 
achievement  was  (-.o9)  before  training,  whereas,  this  correlation  was  improved  significantly  after  training  (r  = 
.51,  p<  .02).  t-test  between  the  two  correlations  was  significance  {t(18)=-8.9,  p>.001 }. 

Nevertheless  the  overall  results  indicated  that  training  teachers  on  instructional  designer's  competencies 
enhanced  their  planning  routine  and  their  students  academic  achievement,  using  true  experimental  designs  like  a 
pretest-posttest  control  group  design  is  recommended  in  future  research. 

Introduction  and  Review  of  literature: 

Instructional  designers  have  hypothesized  that  the  potential  for  learner  achievement  is  enhanced  when 
teachers  practice  Instructional  Designer  Competencies  (IDC)  during  the  planning  of  their  instruction  (Branch, 
Darwazeh,  &  El-Hindi,  1991,1992).  In  fact,  instructional  designers  believe  that  teachers'  planning  routine  is 
enhanced  when  teachers  receive  a  kind  of  training  on  how  to  design  instruction  systematically 
(Branch,  Darwazeh,  &  El-Hindi,  1992;  Branch,  1993,1994;  Darwazeh,  1993;  Earle,  1991 ;  Martin,  1990;  Reiser  & 
Mory,  1991;  and  Wilkerson,  &  Scheffler,  1992)  (See  Figure  1). 
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Pigurel.     Input-output  relationship  between  designing, 
teaching,  and  students'  achievement. 


The  systematic  design  of  instruction,  however,  is  defined  as  a  discipline  concerned  with  decision  making 
and  developing  instruction  through  processes  of  selection,  analysis,  sequencing,  implementation,  management,  and 
evaluation  (Briggs  &  Wager,  1981,  Darwazeh,  1986;  Dick  &  Carey,  1990;  Gagne,  Briggs,  &  Wager,  1992; 
Merrill,  1983;  Reigeluth,  1983,  pp.  7-9).  The  role  of  instructional  designer,  accordingly,  is  defined  as  one  who 
understands  and  practices  the  activities  of  the  instructional  design  science  in  order  to  accomplish  a  specified  purpose 
under  a  certain  condition  (Darwazeh,  Branch,  &  El-Hindi.  1991;  Martin,  1984,  Reigeluth,  1983). 

Some  educators  (e.g.,  Darwazeh,  1993;  Reigeluth,  1983;  pp.  7-9)  have  viewed  the  systematic  design  of 
instruction  as  being  comprised  of  five  major  domains.  They  are:  1)  Instructional  Analysis,  2)  Instructional  Design 
or  Sequence,  3)  Instructional  Development  and  Implementation,  4)  Instructional  Management,  and  5)  Instructional 
Evaluation.  Each  domain  has  certain  activities  in  which  all  together  comprised  the  Instructional  Designer 
Competencies  (IDC)  (  See  Appendix  1 ) 

Based  on  the  concepts  of  Instructional  Design  and  its  activities,  several  models  have  been  drawn  for 
designing  instruction  (e.g.,  Dick  &  Carey,  1990;  Gagne,  et  al.  1992;  Merrill,  1983;  Pratt,  1980;  Reigeluth,  1983). 
Dick  and  Carey's  model,  in  particular,  has  received  a  great  deal  of  attention  by  professionals  specially  who  work  on 
teachers'  education  and  care  about  their  professional  development.  In  some  cases,  this  model  was  used  as  a  guide  for 
training  teachers  on  how  to  plan  for  their  teaching  systematically  (e.g.,  Applefield  &  Earle,  1990;  Branch, 
Darwazeh,  &  El-Hindi,  1991,1992;  Branch,  1993,  1994;Earle,  1985;  1992).  The  current  study  is  one  of  those  cases 
which  used  Dick  and  Carey's  model  for  the  same  purpose  (See  Figure  2). 
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The  assumption  behind  using  Instructional  design  models  for  training  teachers  to  plan  their  teaching  systematically 
is  that:  teachers,  in  fact,  do  engage  in  many  instructional  design  practices  when  they  plan  for  teaching.  Clearly  the 
roles  of  classroom  teachers  are  like  that  of  instructional  designers.  And  taking  on  the  role  of  instructional  designer, 
on  the  part  of  classroom  teachers  would  have  a  great  influence  on  the  quality  of  the  teachers'  professional 
performance,  hence,  on  the  level  of  their  students'  academic  achievement. 

To  verify  this  assumption,  educators  have  raised  two  questions: 

1-  To  what  extent  do  teachers  practice  Instructional  Designer  Competencies  while  they  plan  their  teaching? 

2-  Is  there  a  relationship  between  teachers  who  practice  instructional  designer  competencies  and  their  students' 
academic  achievement? 

The  First  question  was  addressed  by  a  small  number  of 
studies.  These  studies  tried  to  compare  between  teachers  who  had  a  knowledge  or  a  kind  of  training  in  ii  xtional 
design  skills,  and  teachers  who  did  not  have  such  knowledge  or  skills.  Martin,  for  example,  (Martin,  1990) 
compared  between  two  groups  of  teachers:  One  group  consisted  of  five  teachers  who  have  a  formal  educational 
background  ir.  instructional  systems  design  (ISD),  and  the  other  group  consisted  of  five  teachers  who  did  not  have 
such  knowledge.  An  open  and  closed  item  questionnaire  was  sent  to  each  subject  and  follow-up  interviews  were 
conducted.  Teachers  were  asked  about  their  general  planning  practices,  written  planning  procedures,  mental 
planning,  and  how  they  implement  instruction  based  on  their  plans. 

Results  indicated  that  nine  of  the  ten  teachers  use  genera!  ISD  skills  in  planning.  Teachers  reported  that 
they:  analyze  learners'  need  and  abilities,  and  use  objectives  to  guide  the  instructional  process,  specially  for 
selecting  learning  activities  and  evaluations.  Martin  also  found  that  the  teachers  with  ISD  were  more  spec:1c  in 
their  responses  about  the  use  of  instructional  design  for  some  aspects  of  planning,  such  as,  using  hierarchies  and 
taxonomies  to  sequence  instruction,  checking  the  consistency  among  objectives,  learning  activities,  and  evaluation. 
In  addition,  she  found  that  four  of  the  five  ISD  teachers  believed  that  knowledge  of  ISD  has  improved  both  their 
planning  processes  and  their  teaching. 

Reiser  and  Mary  ( 1 99 1 )  have  reached  similar  results  to  the  above  one  when  they  conducted  a  study  to 
examine  the  extent  to  which  systematic  planning  techniques  (SPT)  were  incorporated  into  the  written  and  mental 
teachers  planning  practices.  They  took  two  experienced  teachers:  one  had  received  a  formal  training  in  the  use  of 
those  techniques,  and  the  other  had  not.  A  questionnaire  was  administered  to  the  teachers  and  they  were  interviewed 
and  observed  as  they  went  about  planning  and  implementing  an  instructional  unit. 

Results  indicated  that  the  teacher  who  had  been  trained  to  use  (SPT)  did  employ  them,  Whereas  the 
untrained  teacher  just  adhered  to  the  principle  that  instructional  activities  should  be  planned  with  objectives  clearly 
in  mind.  Reiser  and  Mory  also  found  that  the  trained  teacher  focused  on  a  written  plan  as  well  as  on  mental  one, 
whereas  the  untrained  teacher  focused  on  a  mental  plan  only.  The  trained  teacher  also  put  emphasis  on  the 
consistency  between  the  objectives  and  the  activities  she  used  during  teaching.  In  addition,  the  trained  teacher  spent 
more  time  in  implementing  the  written  plan  than  the  untrained  teacher. 

Branch,  Darwazeh,  &  El-Hindi  (1992)  sustained  the  above  results  in  some  aspects.  They  found  that  the 
planning  activities  of  classroom  teachers  correlate  with  the  practices  of  instructional  design  professionals,  and  this 
correlation  is  influenced  by  the  subject  taugr'..  Sixty-one  public  school  teachers  from  the  northeast  United  States, 
and  a  35-item,  two-  part  questionnaire  were  used  for  the  purpose  of  their  study. 

Recently,  Branch  (1993,1994)  tried  to  specify  which  practices  teachers  do  more  while  they  plan  teaching, 
and  which  ones  they  do  least.  The  general  results  of  his  studies  were  that,  secondary  school  teachers  tend  to  engage 
in  some  systematic  instructional  design  practices  but  are  selective  in  which  instructional  design  practices  they 
routinely  employ.  While  there  is  a  correlation  between  teachers'  planned  activities  and  instructional  design 
practices,  there  are  several  instructional  design  practices  still  beyond  the  realm  of  teachers  manipulation.  The  most 
important  practices  to  all  teachers  were:  determining  course  goals,  breaking  down  curriculum  goals  into  learning 
tasks,  organizing  the  content  of  each  lesson  around  central  themes,  and  making  sure  the  lesson  fits  within  the 
entire  curriculum.  Whereas,  the  least  important  practices  to  teachers  were:  discussing  lesson  plans  with  others, 
establishing  media  selection  criteria,  soliciting  input  from  subject  matter  experts,  and  coordinating  cooperative 
efforts  among  other  teacher. 
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By  talcing  preservice  teachers,  Earle  (1991,  1992)  identified  graduates  (1980-1990)  from  UNCW 
undergraduate  preservice  teacher  program  and  asked  them  to  respond  to  a  four-  part  survey  which  covered 
demographics,  general  information,  yearly  planning,  unit  planning,  and  daily  planning.  The  results  of  his  survey 
were  as  follows: 

a)  81  %  of  teachers  felt  that  a  knowledge  of  ID  processes  had  improved  their  planning. 

b)  The  crucial  elements  of  the  ID  process  were  goals,  learner  analysis,  objectives,  tests,  activities/strategies,  and 
revision  of  instruction.  Some  other  elements  were  considered  helpful,  if  time  allowed,  such  as:  task  analysis, 
classification  of  learning,  instructional  plans,  and  tried-out  instruction. 

c)  Although  teachers  regarded  mental  and  written  planning  forms  as  almost  equai  in  importance  at  the  yearly, 
unitary,  and  daily  levels,  they  favored  mental  planning  form  overall. 

d)  Teachers  in  their  planning  types  deviated  more  from  yearly  plans  and  less  from  unit  and  daily  plans. 

e)  52.6%  of  teachers  do  plan  for  their  instruction  formally,  whereas  47.4%  do  plan  informally. 

Klein  (1991)  also  conducted  a  study  on  preservice  teachers  to  examine  their  success  in  acquiring  and 
applying  principles  of  learning  and  instructional  design.  105  teachers  enrolled  in  a  professional  teacher  preparation 
program  were  taken  for  this  purpose.  They  were  taught  the  essentials  of  learning  and  competency-based  instruction 
and  were  required  to  plan  a  lesson  using  these  concepts. 

Klein  found  that  most  of  the  preservice  teachers,  regardless  of  their  specialization,  were  successful  in 
acquiring  and  using  the  principles  of  learning  and  instructional  design. 

On  the  other  hand,  Wilkerson  &  Scheffler  (1992)  found  an  opposite  result  to  Klien's.  Wilkerson  & 
Scheffler  in  their  study  tried  to  test  the  assumption  that  preservice  teachers  who  are  trained  to  develop  a  lesson 
plan,  will  with  equal  acumen,  be  able  to  implement  that  plan.  Twenty-six  preservice  teachers  in  an  early  childhood 
practicum  developed  and  implemented  a  mathematics  lesson  plan.  The  lesson  plans  were  evaluated  according  to 
specific  criteria  set  forth  in  the  National  Council  of  Teachers  of  Mathematics'  Curriculum  and  Evaluation  Standards 
for  School  Mathematics.  The  implementation  of  each  plan  was  also  assessed  using  a  Likert  Scale  reflective  of 
similar  criteria.  They  found  that  the  correlation  between  the  scores  on  the  lesson  plan  checklist,  and  the  scores  on 
the  observation  checklist  was  not  significant. 

The  finding  suggest  that  it  cannot  be  assumed  that  preservice  teachers  will  automatically  make  the 
transition  from  written  lesson  plan  to  classroom  implementation  of  those  plans. 

Purpose  of  the  study: 

As  we  can  see  from  the  above  results,  teachers  who  have  a  previous  knowledge  or  got  some  training  or  a 
course  in  instructional  designer  competencies,  would  plan  their  instruction  more  systematically  and  accurately  than 
those  who  did  not  have  such  knowledge,  hence  the  quality  of  their  teaching  performance  is  improved. 

But  the  question  that  remains  unanswered  is:  Do  teachers  who  have  a  knowledge  in  instructional  design, 
influence  positively  the  level  of  their  tfuaents'  academic  achievement? 

By  looking  at  the  previous  studies,  none  of  them  tried  to  address  this  issue.  Therefore,  the  purpose  of 
the  current  study  was  to  address  this  question  by  investigating  two  kinds  of  relationships:  1 )  The  relationship 
between  instructional  designer  competencies  and  teachers'  planning  routine,  and  2)  relationship  between  teachers 
who  receive  a  training  in  IDC,  and  their  students'  academic  achievement. 

Research  Questions: 

The  current  study  tried  to  answer  the  following  questions: 
1  -  Does  a  training  on  IDC  have  a  significant  effect  on  in-service  teachers  planning  routine? 

2-  Are  teacher's  practices  of  IDC's  Domains  (Analysis,  Sequences,  Development  and  Implementation, 
Management,  and  Evaluation)  differ  significantly  from  before  to  after  training? 
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3-  Are  teacher's  practices  of  IDC's  Types  (daily,  monthly,  seasonally,  and  yearly)  differ  significantly  from  before  to 
after  the  training? 

4-  Are  teacher's  practices  of  IDC's  Forms  (Written,  Mental,  and  both:  mental  &  written  forms  together)  differ 
significantly  from  before  to  after  training? 

5-  Does  the  training  on  IDC  have  a  significant  effect  on  teachers'  planning  routine  with  regard  to  the  following 
independent  variables: 

a)  Gender. 

b)  Years  of  teaching  experience. 

c)  The  number  of  training  programs  that  the  teachers  have         taken  during  in-service. 

d)  Teachers'  college  specialization. 

e)  The  level  of  learning  stage  that  the  teachers  teach  in. 

6-  What  are  the  IDC  that  teachers  practice  (90%)  or  more  in  their  planning  routine? 

7-  What  are  the  IDC  that  teachers  practice  (10%)  or  less  in  their  planning  routine? 

8-  Does  the  relationship  between  teachers'  planning  routine  and  their  students'  academic  achievement  differ 
significantly  from  before  receiving  the  training  to  after? 

Methodology 

Sample: 

An  entire  class  of  (37)  in-service  government  school  teachers  who  hold  a  bachelor  degree  in  different 
specialization  at  Nablus  district  was  taken  for  this  study.  Those  teachers  were  enrolled  in  a  course  titled 
"Curriculum  Design  and  Development"  at  An-Najah  University  as  a  requirement  of  a  diploma  certificate  in 
educational  training.  They  have  different  number  of  years  of  teaching  experience  ranging  from  3  to  more  than  25 
years  of  experience.  They  also  teach  grades  from  first  through  twelve. 

Seven  teachers  were  excluded  from  analyzing  the  data  because  they  were  not  government  school  teachers 
rather  they  were  either  affiliated  to  private  or  UNRWA  schools.  Thus  the  final  sample  of  this  study  was  (30) 
government  teachers. 

Experimental  Design: 

One  group  pretest-posttest  pseudo-experimental  design  was  used  for  this  study. 

Statistical  Design: 

Several  statistical  designs  were  used  to  analyze  the  data  in  order  to  answer  the  questions  of  this  study. 
They  were: 

1-  t-test  for  dependent  samples  at  (.05)  a  priori  level  of  significance  was  used  to  compare  between  the  overall  mean 
of  teachers'  planning  routine  before  and  after  receiving  the  training  on  IDC. 

2-  Lindquist  Type  1  ANOVA  with  repeated  measures  on  one  factor  was  utilized  separately  for  each  of  the  following 
independent  variables: 

a)  Instructional  Designer  Competencies'  Domains. 

b)  Instructional  Designer  Competencies'  Types. 

c)  Instructional  Designer  Competencies'  Forms 

d)  Gender. 

e)  Years  of  teaching  experience. 

f)  The  number  of  training  program  that  the  teachers  have  taken  during  in-service. 

g)  Categories  of  teachers'  major  specialization. 

h)  Subject-matter  groupings 

I)  The  level  of  learning  stage  that  the  teachers  teach  in. 

3-  A  "Scheffe"  test  was  used  whenever  F-tesl  showed  significance  at  .05  level. 
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4-  The  "pearson  r  correlation"  was  computed  twice  between  the  overall  mean  of  teachers  planning  routine,  and  the 
average  mean  of  their  students  achievement  test:  Once  before  the  training,  and  once  after,  t-test  for  dependent 
samples  at  .05  a  priori  leve!  of  significance  was  also  used  to  compare  between  the  two  correlations. 

Questionnaire: 

1-  Questionnaire  Construction: 

Based  on  Dick  and  Carey's  model  of  the  systematic  approach  of  instruction  (Dick  &  Carey,  1990),  and 
Reigeluth's  classification  of  instructional  process  domains,  and  by  referring  to  the  questionnaire  of  IDC  which  was 
developed  by  Branch  &  Darwazeh  (Branch,  Darwazeh,  &  El-Hindi,  1991 ,  1992),  A  41-item  Likert  scale-type 
questionnaire  of  1-5  points  measured  instructional  designer  competencies  (IDC)  was  constructed.  The  questionnaire 
covered  five  domains:  1)  seven  items  for  analyzing  instruction,  2)  seven  items  for  designing  or  sequencing 
instruction,  3)  nine  items  for  developing  and  implementing  instruction,  4)  five  items  for  managing  instruction,  and 
5)  six  items  for  Evaluating  instruction. 

Another  seven  items  were  added  to  the  questionnaire:  Four  of  them  asked  teacher  about  the  type  of 
planning  they  practice:  yearly,  seasonally,  monthly,  and  daily.  The  rest  of  the  items  have  asked  about  the  form  of 
planning  the  teachers  use:  a  written  form,  mental  form,  or  both:  written  and  mental  forms  together.  Questions 
about  demographic  information  was  also  included  but  excluded  from  this  report  (See  Appendix  1). 

The  reason  behind  measuring  five  domains  of  instructional  design  was  that  teachers  are  expected  to  plan 
for  the  whole  instructional  process  while  they  teach.  And  this  process  of  instruction  is  not  confound  on  teaching 
and  evaluating  students  (Branch,  Darwazeh,  &  El-Hindi,  1991).  In  fact,  this  process  includes  five  domains: 
analyzing  and  sequencing,  developing,  implementing,  managing,  and  evaluating  instruction  (Reigeluth,  1983). 

On  the  other  hand,  the  real  world  of  teaching  tells  us  that  a  lot  of  teachers  have  mastered  some  skills  of 
instructional  design.  Thus,  training  them  on  all  activities  of  instructional  design  is  a  waste  of  time  and  effort.  For 
this  reason,  the  current  came  to  identify  five  sub-measures  of  instructional  design  in  order  to  determine  the  domain 
that  the  teachers  lack  of,  hence  to  give  them  the  training  they  need  in  that  domain.  By  doing  that,  time  and  effort 
will  be  saved  specially  in  this  age  which  is  characterized  by  rapid  technological  and  information  advances. 

2-  Questionnaire  Validation: 

The  questionnaire  was  validated  with  10  experts  in  instructional  design  who  are  located  in  six 
Universities:  Syracuse  University,  The  University  of  North  Carolina  at  Wilmington,  The  University  of  Georgia, 
The  Pennsylvania  State  University,  The  Florida  State  University,  and  An-Najah  National  University.  Also  40  in- 
service  teachers  of  (1-12)  grades  have  read  the  questionnaire  and  made  some  changes  and  comments  on  it  to  be 
understandable  from  teachers'  jargon.  All  the  changes  and  comments  that  have  been  provided  by  those  experts  and 
teachers  were  considered.  Thus,  the  content  validity  of  IDC  questionnaire  was  secured.  On  the  other  hand,  the 
"Cronbach  Alpha  coefficient"  was  computed  for  the  reliability  of  the  questionnaire.  It  was  (.91). 

Procedure: 

The  IDC  questionnaire  was  administered  to  the  sample  of  the  current  study  one  week  before  teachers  start 
to  receive  training  on  IDC.  Teachers,  next  day  of  the  administration,  were  asked  to  design  a  written  plan,  in  the 
way  they  used  to,  for  two  lessons  of  their  regular  subjects  they  teach  in  their  schools,  and  submit  it  to  the  author. 
Then,  they  were  required  to  teach  the  two  lessons  in  one  week  as  needed,  and  to  test  their  students  in  the  following 
week  immediately.  Teachers  who  teach  the  same  subject-mater  to  the  same  grade  were  asked  to  teach  the  same 
lessons  and  use  a  unified  test,  and  to  grade  the  test  according  to  the  unified  model  of  solution.  The  author  was 
reviewing  the  achievement  tests  to  make  sure  that  they  measure  all  concepts,  principles,  procedures,  and  facts  that 
the  two  lessons  included.  This  process  of  teaching  and  testing  took  around  ten  days. 

Immediately  after  that  process  of  teaching  and  testing,  teachers  start  to  receive  training  on  how  to  design 
instruction  systematically.  18  hours  were  devoted  for  this  training  out  of  (48)  hours  put  for  the  whole  course. 
Teachers  were  trained  theoretically  and  practically  in  three  consecutive  weeks.  The  training  based  on  Dick  and 
Carey's  model  for  the  systematic  approach  of  instruction,  and  was  taught  by  the  author  herself,  by  using  Merrill's 
Component  Display  Theory  (  Merrill,  1983).  Basically,  teachers  were  given  generalities  of  each  component  of  the 
Dick  and  Carey's  model  (See  Figure  2  again),  then  examples,  practices,  feedback,  and  a  test. 
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Therefore,  the  practice  section  of  the  training  was  focused  on  involving  teachers  in  the  activities  of  each 
domain  of  instructional  design  which  in  sum  comprised  the  IDC 
questionnaire. 

In  the  domain  of  analysis,  teachers  were  asked  to  engage  in  analyzing  activities  such  as:  analyzing  the 
classroom  environment,  determining  its  constraints  and  facilities,  determining  the  educational  goals,  describing 
students'  learning  ability,  social  economic  class,  academic  aptitude  etc.,  conducting  content  analysis,  determining 
behavioral  objectives,  and  identifying  learning  prerequisites  etc. 

In  the  domain  of  design  (or  sequence),  teachers  were  asked  to  engage  in  designing  activities  such  as: 
sequencing  educational  and  behavioral  objectives  hierarchically,  identifying  relationship  among  topics,  match 
educational  goals  to  performance  objectives;  match  instructional  strategies  to  performance  objectives;  match 
performance  objectives  and  instructional  strategies  to  lessons  content;  match  performance  objectives,  instructional 
strategies,  and  lessons  content  to  test  items  etc. 

In  the  domain  of  development  and  implementation,  teachers  were  asked  to  engage  in  teaching  activities 
such  as:  selecting  or  developing  primary  teaching  strategies;  selecting  or  developing  alternative  teaching  strategies; 
determining  cognitive  strategies  for  supporting  student  learning  such  as  organizers,  questions,  note-taking  etc.; 
selecting  strategies  for  recognizing  individual  differences,  for  motivating  students  learning,  and  for  dealing  with 
student  who  are  above  or  below  average;  identifying  available  and  potential  resources  relevant  to  a  planned 
instructional  episode  such  as  textbook,  periodicals  other  important  references;  selecting  relevant  media  etc. 

In  the  domain  of  management,  teachers  were  asked  to  engage  in  managing  activities  such  as:  outlining 
a  time  line  for  accomplishing  the  lessons  objectives;  coordinating  with  school's  principal,  superintendents, 
administrators,  teachers  and  parents  when  necessary;  consulting  with  the  appropriate  specialists,  supervisors, 
subject  matter  experts,  business  and  industry  groups,  educational  organization,  educational  technology  experts 
when  necessary;  planning  for  classroom  management;  planning  for  dealing  with  disruptive  behavior  in  the 
classroom;  keeping  records  of  students'  progress,  achievement,  attendance,  special  needs  etc. 

And  in  the  domain  of  evaluation,  teachers  were  asked  to  engage  in  evaluating  activities  such  as: 
developing  achievement  tests;  developing  a  plan  for  formative  evaluation;  developing  a  plan  for  remedial 
instruction  or  enrichment  activities;  developing  a  plan  for  summative  evaluation  etc. 

We  should  note  here  that  teachers  were  required  to  submit  a  written  report  for  each  domain's  activities. 

Then,  a  take-home  exam  test  was  administered  to  the  teachers  by  giving  them  a  new  written  lesson,  and 
asking  them  to  do  all  the  above  activities  that  have  learned  and  practiced,  but  now  by  using  a  new  lesson. 

At  the  end  of  the  eighteen  hours  of  training,  the  same  subject-matter  groups  of  teachers  were  asked  to 
submit  a  written  plan  for  another  two  lessons  for  the  second  teaching.  Teachers  were  reminded  to  pull  them  out 
from  the  same  unit  that  they  taught  before  the  training,  but  this  time  they  should  follow  in  their  planning  the 
principles  they  have  learned  and  practiced  during  training  processes. 

One  week  later,  teachers  were  asked  to  teach  the  two  lessons  in  one  week  as  needed,  and  to  test  their 
students  in  the  following  week  immediately.  As  it  was  in  the  first  time,  teachers  who  teach  the  same  subject- 
mater  to  the  same  grade  were  asked  to  teach  the  same  lessons  and  use  a  unified  test  and  to  grade  the  test  according 
to  the  unified  model  of  solution.  The  author  was  reviewing  the  achievement  tests  to  make  sure  that  they  measure 
all  concepts,  principles,  procedures,  and  facts  that  the  two  lessons  included.  This  process  of  teaching  and  testing 
took  also  around  ten  days. 

After  that  process  of  teaching  and  testing,  IDC's  questionnaire  was  administered  to  the  teachers  for  the 
second  time,  two  weeks  after  receiving  the  training.  Thus,  four  averages  of  scores  were  collected:  Teachers  average 
scores  on  the  questionnaire:  one  before  the  training  and  one  after.  Students  average  scores  on  the  achievement 
tests:  one  before  their  teachers  received  the  training,  and  one  after. 

It  should  be  noted  here  that  teachers  who  teach  the  same  subject-matter  to  the  same  grade,  and  form  a 
group  of  three  persons  or  more  were  only  considered  for  computing  the  correlation  between  their  scores  on  the 
questionnaire,  and  their  students'  test  scores.  The  aim  of  this  process  was  to  reduce  the  variation  among  teachers 
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who  teach  different  subjects  to  different  grades.  Accordingly,  the  computation  of  the  correlation  has  been  done  only 
on  19  teachers  who  formed  groups  with  respect  to  the  subject-matter  and  grade.  They  came  in  an  arbitrary  way  as 
follows:  4  teachers  teach  Science  to  tenth  grade,  4  teachers  teach  Arabic  to  the  twelve  grade,  4  teachers  teach 
English  language  to  the  tenth  grade,  4  teachers  t.:ach  Arabic  to  the  second  grade,  and  3  teachers  teach  History  and 
Geography  to  the  twelve  grade  (See  Table  6). 

Results 

The  results  of  this  study  were  the  following: 

1-  t-test  for  dependent  samples  showed  a  significant  enhancement  in  teachers'  planning  routine  from  before 
receiving  the  training  on  Instructional  Designer  Competencies  (M  =  3.7)to  after  (M  =  4.10);  (t(29)  =  -2  86, 
p<.0077}.  (See  Table  1). 

2-  A  5x2  factorial  analysis  of  variance  for  repeated  measure  design  with  two  factors:  One  for  between- 
subjects  (Teacher  planning  domains:  analysis,  design,  devolopment  and  implementation,  management,  and 
evaluation),  and  one  for  within-subjects  (Before  training  &  After  training)  showed  a  significant  main  effect  for 
between-subjects  (F(4:145)  =  3.15,  p<.01 },  and  for  within-subjects  (F(l:  145)  =  28.56,  p>.0001 },  but  did  not 
show  a  significant  interaction. 

The  Post-hoc  ANOVA  by  using  a  "Scheffe"  test  at  (.05)  a  priori  level  of  significance  failed  to  show  any 
differences  among  domains.  On  the  other  hand,  it  showed  differences  for  within  subjects  which  indicated  that 
teachers'  planning  routine  was  enhanced  from  before  training  to  after  in  all  domains  except  the  implementation  one 
(See  Table  2:A  &  2:B). 

3-  A  4x2  factorial  analysis  of  variance  for  repeated  measure  design  with  two  factors:  One  for  between- 
subjects  (Teacher  Planning  Types:  daily,  monthly,  seasonally,  and  yearly),  and  one  for  within-subjects  (Before 
training  &  After  training)  showed  a  significant  main  effect  for  only  between-subjects 

(F(3:116)  =  5.41),  p<.001). 

The  Post-hoc  ANOVA  by  using  a  "Scheffe"  test  at  (.05)  a  priori  level  of  significance  indicated  that  the 
annual  planning  (M=4.93)  was  practiced  better  than  the  seasonally  planning  (M=4.45)  and  the  monthly  planning 
(M=4.35),  but  not  than  the  daily  one  (M=4.73)  (See  Table  3:A  &  3:B). 
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4.         A  3x2  factorial  analysis  of  variance  for  repeated  measure  design  with  two  factors:  One  for  between- 
subjects  (Teacher  planning  forms:  written,  mental,  and  both,  written  and  mental  forms  together),  and  one  for 
within-subjects  (Before  training  &  After  training  )  showed  a  significant  main  effect  for  between-subjects  {F(2:87)  = 
4.64,  p<.01 },  and  for  within-subjects  (F(l:87)  =  5.63,  p>.01 },  but  did  not  show  a  significant  interaction. 

The  Post-hoc  ANOVA  by  using  a  "Scheffe"  test  at  (.05)  a  priori  level  of  significance  indicated  that  the 
written  form  (M=4.25)  was  practiced  better  than  the  mental  and  written  form  together  (M=3.68),  but  not  than  the 
mental  form  (M=3.81).  For  the  within-subjects,  Scheffe  test  indicated  that  the  only  mental  form  was  significantly 
enhanced  from  before  training  to  after  (M  =3.43  vs.4.2)  (See  Table  4:A  &  4:B). 

5-  Lindquist  Type  i  ANOVA  with  two  factors  was  utilized  separately  for  each  of  these  independent 
variables:  a)  gender,  b)  years  of  teaching  experience,  c)  number  of  training  programs  that  the  teachers  had  taken 
during  in-service,  d)  teachers'  college  specialization,  e)  subject  matter  groupings,  0  the  level  of  leaning  stage  that 
the  teachers  teach  in. 

The  analysis  revealed  the  following  results: 

a)  There  was  no  significant  main  effect  between-subjects  for  each  of  the  nbove  independent  variables. 

b)  There  was  a  significant  main  effect  for  within-subjects,  the  repeated  measure,  ranging  from  (.01)  to 
(.0007)  level  of  significance  for  each  analysis  of  the  above  independent  variables.  It  indicated  that  the  teachers 
planning  routine  was  enhanced  significantly  after  receiving  the  training  (See  Table  5). 

c)  There  was  no  significant  interaction  between  the  repeated  measure  and  each  of  the  above  independent 
variables  except  for  the  learning  stage  (p<.007).  This  interaction  indicated  that  the  elementary  schools  teachers 
planning  routine  was  increased  after  training,  whereas  the  teachers  planning  routine  who  teach  all  grades  (1-12)  was 
decreased  and  vice  versa  before  training  . 

6-  The  items  that  had  been  practiced  90%  or  more  as  follows: 

Question  #  26.    Develop  a  time  line  for  accomplishing  the  course  goais  (M  =  4.86  ). 
Question  #  34.    Prepare  records  that  document  student  progress,  achievement  attendance,  or  special 
needs  (M  =  4.83). 

Question  #  5.      Analyze  the  learning  task  in  order  to  identify  the  primary  and  secondary  content  that  I 
plan  to  teach  (M  =  4.60). 

Question  #  32.    Develop  an  instructional  management  plan  to  make  sure  that  events  take  place  as  they 
were  planned  (M  =  4.6) 

Question  #  29.    Design  an  evaluation  plan  to  determine  students'  strengths  and  weaknesses  in  mastering 
the  performance  objectives  (M  =  4.56). 

Whereas,  the  items  that  had  been  practiced  (10%)  or  less  specially  as  follows: 

Question  #  4.      Analyze  student  history  and  characteristics  (M  =  2.86). 

Question  #  21 .    Select  the  media  that  are  relevant  to  implementing  the  course  (M  =  3.06). 

Question  #  28.    Consult  with  resources,  specialists:  administrators,  supervisors,  content  experts,  or 

people  from  business  and  industry,  while  I  plan  for  the  course  (M  =  3.26). 
Question  #  27.    Coordinate  activities  and  events  with  school  administrators,  other  teachers,  or  parents  to 

manage  or  run  the  course  smoothly  (M  =  3.36). 

7-  The  "Pearson  correlation  coefficient"  between  teachers'  planning  routine  and  their  students'  academic 
achievement  was  almost  zero  (r  =  -.09)  before  training,  whereas  this  correlation  was  improved  significantly  after 
training  (r  =  .51,  p<  .02).  t-test  between  the  two  correlations  was  (t(19)  =  -8.9,  p>.001 }  (See  Table  6). 

Limitation  of  the  study 

Before  discussing  the  results  of  this  study,  it  is  worthwhile  to  bear  in  mind  that  the  experimental  design 
of  this  study  was  a  pseudo-experimental  design  which  might  threaten,  somewhat,  the  internal  validity  of  this  study 
such  as  history,  maturation,  testing,  and  mortality  variables.  At  the  same  time,  considering  that  the  study  dealt 
with  teachers  aged  over  thirties,  and  submit  them  to  a  short  period  of  time  of  training  (three  weeks),  the  effects  of 
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maturation  and  history  are  relatively  reduced  if  it  is  not  controlled.  In  addition,  the  mortality  extraneous  variable  is 
controlled  by  talcing  the  same  number  of  teachers  before  and  after  the  training.  Nevertheless,  we  still  recommend 
other  researchers  to  use  true  experimental  design  like  a  pretest-posttest  control  group  design  in  their  future  studies. 

Discussion 

The  overall  mean  indicated  that  training  teachers  on  instructional  designer  competencies  enhanced 
significantly  (p>.0007)  their  teaching  planning  routine,  hence,  their  students'  academic  achievement.  This  results 
showed  clearly  the  importance  for  the  teachers  to  use  the  systematic  approach  of  instruction  while  they  plan  for 
teaching.  This  result  is  in  consistence  with  the  results  of  most  of  previous  studies  that  had  investigated  the  same 
theme  and  reached  to  similar  results  (e.g.,  Branch,  Darwazeh,  &  El-Hindi,  1992,  Martin,  1990;  Reiser  &  Mory, 
1991;  Earle,  1991,  1992). 

The  most  interesting  finding  of  this  study  was  the  significant  correlation  (r  =  .51,  p<  .02)  between 
teachers  who  received  training  on  IDC  and  their  students'  academic  achievement.  This  correlation  was  zero  before 
training,  and  became  moderate  and  significant  after  training  (See  Table  6). 

This  finding  showed  the  importance  and  effectiveness  of  acquiring  IDC  skills  not  just  for  enhancing  teachers' 
planning  routine,  but  also  to  improve  their  students'  academic  achievement.  Since,  this  is  the  only  study  that  have 
investigated  this  kind  of  relationship,  the  author  strongly  recommends  other  researchers  to  conduct  more  studies  to 
validate  this  result.  The  author  also  recommends  the  ministry  of  education  in  Palestine  and  in  any  country  to 
submit  their  teachers,  who  still  do  not  have  any  knowledge  or  skills  in  instructional  design,  to  a  training  program 
or  give  them  a  course  related  to  it. 

With  respect  to  the  types  of  planning,  results  of  this  study  showed  that  the  annual  planning  was 
practiced  most,  whereas,  the  monthly  and  seasonally  planning  were  practiced  least  with  no  differences  between  the 
annual  and  daily  planning.  This  could  be  interpreted  that  Palestinian  teachers  do  care  to  plan  for  either  the  whole 
curriculum  or  each  lesson  of  it,  but  not  for  each  course  or  unit  which  was  expressed  in  a  seasonally  or  monthly 
plan.  This  result  contrasts,  somewhat,  to  the  Earle's  results  (1991,1992)  which  indicated  that  teachers  deviated 
more  from  yearly  plans  and  less  from  unitory  (monthly)  and  daily  plans.  This  could  also  be  interpreted  by  the 
differences  might  be  existed  between  the  Palestinian  and  the  United  Sates'  educational  systems.  Cross  culture 
studies  are  greatly  recommended  on  this  issue. 

With  respect  to  the  planning  forms,  the  results  indicated  that  the  written  plan  was  practiced  most, 
whereas,  the  written  and  mental  plan  form  together  was  practiced  least  with  no  differences  between  the  written  and 
mental  forms.  This  result  is  in  agreement  with  Reiser  and  Mory's  ones  (1991)  which  indicated  that  trained  teachers 
on  using  the  systematic  planning  techniques  have  focused  on  a  written  plan  as  well  on  the  mental  one. 

Other  interesting  finding  of  the  current  study  was  that  the  mental  form  of  planning  was  the  only  form 
which  improved  significantly  from  before  training  to  after.  This  could  show  the  training  process  succeeded  in 
attracting  teachers'  attention  to  the  systematic  approach  of  instruction  and  letting  them  think  about  it. 

Other  surprising  finding  of  this  study  was  the  significant  interactions  between  the  students  learning  stage 
and  the  repeated  measure  (p<.007).  This  interaction  indicated  that  the  elementary  schools  teachers'  planning  routine 
was  increased  after  training,  whereas,  teachers'  planning  routine  who  teach  all  grades  (1-12)  was  decreased  and  vice 
versa  before  training. 

This  result  could  be  interpreted  by  saying  that  teachers  who  teach  all  grades  may  get  confused  when  they 
plan  for  all  grades  simultaneously.  Whereas,  teachers  who  teach  in  a  defined  stage  know  exactly  what  they  are 
going  to  do.  Why  the  elementary  stage  not  other  stages?  The  reason  could  be  that  teachers  of  elementary  stage  can 
transfer  the  principles  of  instructional  design  more  easily  than  those  who  teach  in  the  other  stages  like  preparatory 
or  secondary;  because  they  deal  with  less  complexity  of  subject-matter  than  the  other  two  stages.  However, 
considering  that  there  was  one  teacher  in  the  cell  of  all  grades  (See  table  5),  we  can  not  trust  this  result.  Thus, 
further  research  is  recommended  to  take  bigger  sample  in  each  stage. 

Before  signing  off,  it  is  worthwhile  to  note  that  the  general  (F)  showed  a  signicant  main  effect  among 
teacher's  practices  of  the  five  domains,  though  the  "Scheffe"  test  could  not  reveal  such  significance.  The  means 
of  design  and  analysis  domains  were  higher  than  the  implementation,  management,  and  evaluation  ones 
respectively  (M  =  4.06  vs.  4.01  vs.  3.76  vs.  3.76  vs.  3.79).  These  means  could  tell  us  that  teachers  have  already 
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had  some  knowledge  and  skills  in  lanning  instruction.  Thus,  it  is  very  beneficial  to  measure  teachers'  planning 
routine  in  each  domain  rather  than  measuring  it  as  a  whole.  This  process  could  help  us,  as  instructional  designers, 
to  give  teachers  the  training  they  need  in  the  domain  they  lack  of. 

Accordingly,  instructional  designers  are  recommended  to  work  more  on  IDC's  questionnaire  and  its  five 
domains,  so,  it  can  be  used  as  a  diagnostic  tool  for  identifying  the  domain  that  the  teachers  lack  of.  In  this  way 
we  can  save  a  lot  of  money  and  effort  when  we  give  teachers  the  training  they  need  in  the  domain  they  lack  of, 
rather  to  give  them  training  in  all  domains  they  have  already  mastered  some  of  them. 

In  sum,  instructional  designer  competencies  are  very  important  to  acquire  by  teachers  in  order  to  improve 
the  quality  of  their  teaching,  hence,  the  level  of  their  students'  academic  achievement.  More  research  is  greatly 
recommended  to  investigate  the  relationship  between  teachers  who  had  instructional  designer  skills  and  their 
students'  academic  achievement  by  using  true  experimental  designs  like  a  pretest-posttest  control  group  design. 
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Tables  1  -  6  and  Appendix  1  appear  below. 
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Table  1.  t-test  for  Teachers  Planning  Routine  Before  and  After  receiving  training  in  Instructional 
Designer  Competencies. 
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Tabic  2-A       Lindquist  TypetANOVA  Summary  Tabic  for  Instructional  Designer  Competencies' 
Domain. 
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Tabic  2:8  Teachers  Means,  Standard  Deviations,  and  Samples  for  Instructional  Designer  Competencies' 
Domain  Before  and  After  Training. 
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Table  3:A  Undquist  Type  1  ANOVA  Summary  Tabic  for  iDsOucdona!  Designer  Competencies'  Type. 
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Table  3:8  Teachers  Means,  Stindard  Deviations,  and  Samples  for  Instructional  Designer  Competencies' 
Type  Before  and  After  Training. 
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Tabic  4:A  Lindquist  Type  i  ANOVA  Samrnary  Table  for  Instructional  Designer  Competencies  Form. 
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Table  4:B  Teachers  Means,  Standard  Deviations,  and  Samples  for  Instructional  Designer  Competencies' 
Form  Before  and  After  Training. 
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Table  6:  "Pearson  r"  between  teachers  means  on  IDC  questionnaire  and  their  students'  academic 
achievement  Before  and  After  Training 
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Appendix  1. 

Instructional  Designer  Competencies'  Questionnaire. 

The  items: 

As  I  plan  for  teaching,  I  do  the  following: 

1 .  Analyze  the  school  setting  in  which  the  course  would  be  delivered. 

2.  Review  the  goals  for  my  course  that  are  stated  in  curriculum  guides  or  course  manuals. 

3.  Determine  the  constraints  that  I  may  face  durii.g  teaching. 

4.  Analyze  student  history  and  characteristics,  including  past  academic  patterns  and  levels,  age  and  maturity, 
special  aptitudes  or  disabilities,  and  socioeconomic  characteristics. 

5.  Analyze  the  learning  task  in  order  to  identify  the  primary  and  secondary  content*  that  I  plan  to  teach  in  the 
course. 

6.  Organize  the  course  content,  including  the  concepts,  rules,  procedures,  and  facts,  hierarchically  from  simple 
to  complex. 

7.  Determine  how  items  of  the  course  content  are  related  to  each  other. 

8.  Identify  the  prerequisite  learning  or  characteristics  that  students  should  have  in  order  to  learn  what  I  plan  for 
the  course. 

9 .  Specify  performance  objectives  for  each  lesson  in  the  course,  or  decide  what  the  objectives  will  be. 

1 0.  Organize  performance  objectives  hierarchically  from  simple  to  complex. 

1 1 .  Match  the  course  goals  to  performance  objectives. 

1 2.  Select  or  produce  the  teaching  methods  that  are  essential  for  the  course. 

1 3.  Select  or  produce  the  teaching  methods  that  will  be  used  as  alternatives  for  the  course. 

1 4.  Match  teaching  methods  to  the  learning  stated  in  the  performance  objectives. 

1 5.  Match  performance  objectives  and  teaching  methods  to  the  course  content. 

1 6.  Determine  the  aids  to  thinking  that  are  needed  to  support  learning  by  the  student;  such  as  organizers,  related 
questions,  summaries,  note  taking,  analogies,  stories,  or  outlines. 

1 7.  Select  or  develop  methods  for  recognizing  differences  among  the  students. 

1 8.  Select  or  develop  methods  for  motivating  students  to  engage  in  learning  the  course. 

1 9.  Make  plans  for  using  methods  of  reinforcement  during  teaching. 


"Course  Content"  when  used  in  any  item  refers  to  all  the  concepts,  rules, 
procedures,  and  facts  that  might  be  the  intended  learning  for  a  course. 
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20.  Determine  the  available  textbooks,  reference  materials,  periodicals,  or  other  resources  that  are  suitable  to  a 
planned  course. 

2 1 .  Select  the  media  that  are  relevant  to  implementing  the  course. 

22.  Plan  for  facilitating  verbal  and  non-verbal  interactions  between  the  teacher  and  students. 

23.  Develop  the  achievement  tests  that  would  be  used  for  the  whole  course. 

24.  Match  performance  objectives,  teaching  methods,  and  the  content  of  the  course  to  the  test  items  that  are 
being  considered. 

25 .  Develop  or  adopt  evaluation  procedures  besides  achievement  tests  to  evaluate  other  student  learning 

26.  Develop  a  time  line  for  accomplishing  the  course  goals,  with  a  consideration  given  for  holidays  and  other 
interruptions  of  the  instruction. 

27.  Coordinate  activities  and  events  with  school  administrators,  other  teachers,  or  parents  to  manage  or  run  the 
course  smoothly. 

28.  Consult  with  resources,  specialists  (administrators,  supervisors,  subject  matter  experts,  or  people  from 
business  and  industry),  while  I  plan  for  the  course. 

29.  Design  an  evaluation  plan  to  determine  students'  strengths  and  weaknesses  in  mastering  the  performance 
objectives. 

30.  Design  a  plan  for  remedial  or  enrichment  instructional  activities. 
3  1 .  Design  an  evaluation  plan  for  assigning  final  grades. 

32.  Develop  an  instructional  management  plan  to  make  sure  that  events  take  place  as  they  are  planned. 

33.  Make  plans  for  dealing  with  disruptive  behaviors  in  the  classroom. 

34.  Prepare  records  that  document  student  progress,  achievement,   attendance,  or  special  needs. 


35. 

I  use  the  above  activities  yearly  when  I  plan  for  the  whole  academic  year. 

36. 

I  use  the  above  activities  seasonally  when  I  plan  for  the  whole  course. 

37. 

I  use  the  above  activities  monthly  when  I  plan  for  each  unit  of  the  course. 

38. 

I  use  the  above  activities  daily  when  I  plan  for  each  lesson  of  the  course. 

39. 

I  plan  for  these  activities  mentally. 

40. 

I  plan  for  these  activities  in  written  form. 

41. 

I  plan  for  these  activities  both:  mentally  and  in  written  form. 

Note:     Analysis  Domain  =Items  #1+2+3+4+5+8+9 

Design  or  Sequence  Domain  =Items  #6+7+10+11  +  14+15+24 

Development  &  Implementation  Domain  =Items  #12+13+16+17+18+19+20+21+22 

Management  Domain  =Items  #27+28+32+33+34 

Evaluation  Domain  =Items  #23+25+26+29+30+3 1 
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Abstract 

Cognitive  task  analysis  grew  out  of  efforts  by  cognitive  psychologists  to  understand  problem- 
solving  in  a  lab  setting.  It  has  proved  a  useful  tool  for  describing  expert  performance  in  complex 
problem  solving  domains.  This  review  considers  two  general  models  of  cognitive  task  analysis 
and  examines  the  procedures  and  results  of  analyses  in  three  domains.  From  the  standpoint  of 
technique,  cognitive  task  analysis  can  and  should  be  integrated  into  systematic  instructional  design. 
However,  true  integration  will  require  instructional  designers  to  reexamination  their  theories  of 
learning  with  an  eye  toward  adopting  the  learning  constructs  developed  by  cognitive  psychologists. 
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Cognitive  Task  Analysis:  Implications  for  the  Theory  and  Practice  of  Instructional  Design 

Introduction 


Task  analysis  as  practiced  within  the  world  of  instructional  design  typically  results  in  a 
linear  description  of  the  job  process  and/or  hierarchical  orderings  of  the  intellectual  skills  required 
to  achieve  the  task.  These  constructs  are  particularly  useful  for  defining  observable  tasks  easily 
subject  to  top  down  (or  bottom  up)  analysis.  When  the  object  of  training  is  to  move  people 
toward  expert  performance  in  a  complex  problem  solving  task,  an  instructional  designer  relying 
entirely  on  task  analysis  methodologies  from  the  instructional  design  literature  may  be  at  a  loss. 
However,  useful  task  analysis  models  and  techniques  have  been  developed  by  cognitive  scientists 
working  in  the  areas  of  knowledge  engineering,  ergonomics,  and  cognitive  measurement.  These 
techniques  generally  focus  on  illuminating  the  "covert  heuristic"  (Wilson  and  Cole,  1990)  used  by 
experts  to  solve  problems,  and  resuit  in  a  description  of  an  expert's  mental  model  of  the  problem. 

Cognitive  task  analysis  (CTA)  techniques  were  first  used  by  cognitive  scientists  studying 
human  cognition  in  a  lab  setting,  typically  by  having  subjects  (college  students)  work  through 
domain-free  problems  such  as  "The  Tower  of  Hanoi"  or  "Missionaries  and  Cannibals".  Of 
interest  to  instructional  designers,  however,  is  CTA's  usefulness  as  a  methci  for  describing  the 
performance  of  people  who  solve  difficult  problems  for  a  living.  Funke  (1!>91)  identified  six 
features  common  to  such  problems: 

1)  Intransparency — only  some  aspects  lend  themselves  to  direct  observation;  often  one  must 
infer  an  underlying  state  from  observational  data.  Alternatively,  the  problem  can  be  fully  assessed 
in  principle,  but  it  involves  so  many  variables  that  by  necessity  a  few  relevant  ones  must  be 
chosen; 

2)  Poiytely — multiple  goals; 

3)  Complexity  of  situation — the  problem  involves  a  large  number  of  control  processes  and 
regulatory  features; 

4)  Connectivity  of  variables — changes  in  one  variable  affects  the  status  of  other  variables; 

5)  Dynamic  developments — the  problem  situation  may  suddenly  change  for  the  worse,  forcing 
a  problem  solver  to  act  immediately  under  time  pressure; 

6)  Time-delayed  effects — not  every  action  show  immediate  results. 

Three  occupations  discussed  extensively  in  the  literature  as  subjects  for  CTA:  air  traffic 
controllers  (Means,  1993),  electronics  specialists  troubleshooting  avionics  equipment-testing 
workstations  (Lajoie  &  Lesgold,  1992;  Lesgold  &Lajoie,  1991;  Lesgold,  Lajoie,  Logan,  &  Eggan, 
1990;  and  Means,  1993),  and  nuclear  power  plant  operators  (Roth  &  Woods,  1992;  Roth,  Woods, 
and  Pople,  1992),  reflect  these  qualities. 

Given  Funke's  characterization,  among  the  problem-solvers'  most  important  skills  will  be 
metacognitive  strategies  for  selecting  relevant  information,  prioritizing  and  revising  goals,  and 
working  between  multiple  versions  of  the  problem  representation.  For  example,  a  practitioner 
may  keep  current  and  projected  models  in  mind.  A  CTA  approach  provides  a  framework  for 
identifying  and  integrating  skills  such  as  these  into  the  overall  task  description. 

This  review  of  the  literature  considers  the  roots  and  practice  of  CTA,  with  the  general  goal  of 
elucidating  and  perhaps  expanding  the  notion  of  task  analysis  in  instructional  design.  The  first 
section  of  the  paper  summarizes  the  underlying  theory  of  CTA,  focusing  on  research  in  problem- 
solving  and  expertise.  The  following  sections  review  the  literature  on  CTA  models  and  techniques 
and  examine  weaknesses  of  CTA.  Finally,  implications  of  the  technique  for  the  theory  and 
practice  of  instructional  design  are  considered. 

Underlying  theory 

Problem-solving 

Problem-solving  theory  developed  within  the  information-processing  view  of  cognition  (see, 
Reimann  &  Chi,  1989  for  a  review).  The  terminology  from  this  strand  of  the  literature  is  also 
widely  used  in  describing  CTA.  Initially  the  solver  internally  represents  the  problem  in  terms  of 
its  objects,  actions  that  can  be  taken  on  objects,  strategies  that  can  be  used  to  work  on  the 
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problem,  and  constraints  on  objects,  actions,  and  strategies.  This  original  representation  of  the 
problem  as  given  is  called  the  initial  state,  which  evolves  into  subsequent  problem  states  as 
operators  are  applied  by  the  solver  until  the  goal  state  is  reached.  The  set  of  all  possible  problem 
states  and  all  possible  combinations  of  operators  is  the  problem  space.  By  definition,  problem 
solving  is  the  process  of  picking  a  path  through  the  problem  space  from  the  initial  state  to  the 
goal  state. 

Expertise 

When  conducting  a  task  analysis,  the  problem  representation  of  an  expert  (or  a  novice-expert 
comparison)  is  of  particular  interest.  But  in  complex  domains  expert  problem-solving  becomes 
inextricably  linked  with  domain  knowledge  (Reimann  &  Chi,  1989,  Bedard  &  Chi,  1992).  The 
problem  representations  and  operators  used  by  experts  are  specific  to  their  domain,  and  cannot  be 
easily  identified  outside  the  domain.  Expert  performance  is  characterized  by  other  consistent 
features.  Experts'  domain  knowledge  is  broader,  more  interconnected,  and  better  organize  ".  They 
learn  to  quickly  recognize  recurring  patterns  in  their  domains  (Bedard  &  Chi,  1992).  As  expertise 
grows,  procedural  knowledge  becomes  more  comprehensive  and  automatic,  freeing  up  processing 
capacity.  (Means,  1988)  This  process  can  lead  to  a  dissociation  between  verbal  knowledge  and 
performance  as  experts  may  ,iot  be  able  to  easily  describe  what  they  do  (Sanderson,  1989). 
Another  component  of  experts'  problem-solving  capacity  is  increased  memory  for  information  in 
their  problem  domain  (Anderson,  1993).  Experts  tend  to  view  problems  by  focusing  on 
underlying  principles  instead  of  surface  features  and,  as  opposed  to  novices,  will  impose  structure 
on  ill-defined  problems.  Finally,  when  following  a  means-end  problem  solving  strategy  (the 
process  of  iteratively  setting  and  revising  intermediate  goals  to  reach  a  targeted  end-goal),  experts 
follow  more  efficient  paths  (Bedard  &  Chi,  1992). 

Implications 

The  implications  for  cognitive  task  analysis  are  important.  The  primary  goal  of  CTA  is  to 
reveal  the  problem  representations  and  operators  of  the  solver  (operations  include  actions  and 
strategies).  But  because  domain  knowledge  and  expert  performance  are  linked,  the  analyst  must 
acquire  enough  domain  expertise  to  ask  the  right  questions.  The  finding  that  experts  manipulate 
domain  data  in  higher-level  "chunked"  form  is  particularly  important  because  people  may  be  unable 
to  access  subordinate  elements  of  chunked  information  easily  from  memory.  In  the  same  vein, 
cognitive  strategies  are  generally  not  accessible  to  domain  practitioners  in  the  abstract,  instead 
tending  to  be  buried  features  of  particular  situations  that  are  automatically  applied  when 
appropriate.  Finally,  experts'  domain-specific  patterns  are  another  potential  subject  of  inquiry  for 
CTA. 


Models  of  Cognitive  Task  Analysis 

Most  practitioners  treat  CTA  as  a  fluid  concept  with  the  general  practical  goal  of  identifying 
an  expert's  metacognitive  domain-organizing  structures,  and  in  particular  his  problem 
representation,  whether  it  be  propositional  or  an  internal  visual  representation.  (Wilson,  *989; 
Nelson,  1990).  The  actual  procedure  may  in  some  cases  focus  on  deriving  the  implicit  rule  base 
used  to  solve  a  problem  (Ohlosson,  1990).  The  main  focus  may  also  be  on  determining  the  way 
the  expert  organizes  domain  knowledge,  resulting  in  representations  of  his  schemata,  semantic 
webs,  or  mental  representations  of  the  problem  state  (Means,  1993;  Lesgold  et  al.,  1990;  Nelson, 
1990).  However,  at  least  two  researchers  have  proposed  formal  models  of  cognitive  task  analysis. 

Gardner  (1985)  defines  the  purpose  of  CTA  as  identifying  the  performance  components, 
knowledge  structures,  and  metacognitive  knowledge  underlying  a  task.  Performance  components 
are  the  automatic  mental  processes  underlying  all  cognition,  such  as  encoding,  inference,  response, 
and  performing  operations  on  internal  representations.  Knowledge  structures  are  the  network  of 
propositions  and  rules  that  form  domain  knowledge.  Metacognitive  knowledge,  in  Gardner's  view, 
is  the  collection  of  overt  mental  strategies  people  apply  to  a  task  to  control  higher-order  planning, 
such  as  breaking  the  problem  into  parts. 

Roth  and  Woods  (1989)  define  cognitive  task  analysis  as  a  two  part  process.  The  goal  of  the  first 
stage  is  to  define  competent  performance  of  the  task,  or  to  create  a  model  of  the  problem-solving 
environment.  The  resulting  competence  model  specifies  "what  people  must  be  able  to  do  to 
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accomplish  the  task:  what  kinds  of  problems  they  must  solve,  what  must  they  know  and  how 
must  they  use  this  knowledge  to  solve  problems;  what  knowledge  must  be  accessed  to  select 
goals,  form  intentions  to  act,  to  monitor  and  adapt  plans"  (p.  246).  The  competence  model  defines 
the  parameters  of  the  rest  of  the  CTA,  pointing  the  analyst  toward  the  areas  of  the  task  that  require 
study.  Roth  and  Woods  also  believe  that  by  first  defining  a  competence  model,  the  analyst  is  more 
likely  to  correctly  identify  sources  of  poor  performance.  The  second  stage  of  analysis  derives  a 
performance  model  which  specifies  how  practitioners  actually  perform  the  task.  It  is  in  this  phase 
that  domain  specific  knowledge  is  elicited  through  a  variety  of  interviewing  and  observational 
techniques. 

These  two  models  describe  CTA  at  completely  different,  but  compatible  levels,  which  can  be 
described  in  Roth's  and  Woods'  own  terms.  Gardner's  conception  can  be  seen  as  part  of  a 
performance  model  of  CTA,  for  which  Roth's  and  Wood's  conception  serves  as  the  broader 
competency  model. 

Cognitive  Task  Analysis  Procedures 

While  investigators  have  pursued  a  rich  and  varied  group  of  strategies  to  perform  CTA's,  the 
procedures  can  generally  be  grouped  into  three  categories.  These  are:  1)  methods  for  using 
domain  knowledge  to  structure  the  analysis,  2)  focus  problem  development,  and  3)  knowledge 
elicitation. 

Acquiring  domain  knowledge 

Before  analysts  can  get  very  far,  they  must  learn  enough  about  the  domain  to  be  able  to  ask 
the  right  questions  about  how  people  do  the  task  and  to  be  able  to  understand  the  answers.  The 
objective  is  to  get  a  preliminary  sense  of  "the  basic  concepts,  procedures,  range  of  problems  and 
sources  of  complexity  in  the  task"  (Roth  &Woods,  p.250)  —  to  develop  a  competency  model  that 
serves  to  structure  the  rest  of  the  analysis. 

As  in  any  task  analysis,  one  starts  with  the  "official"  version  of  the  task  by  reviewing  any 
existing  materials  that  describe  it.  Beyond  examining  documentation,  a  typical  approach  at  this 
stage  is  to  interview  several  people  who  perform  it.  However,  these  discussions  can  tend  to  focus 
on  isolated  details  of  the  task,  often  producing  several  unrelated  versions  or  a  skewed  picture  of  its 
true  dimensions.  Roth  and  Woods  recommend  having  a  domain  expert  prepare  and  deliver  an 
overview  presentation  first,  then  moving  into  structured  interviews  to  add  detail.  A  second 
approach  is  to  leverage  expert  knowledge  to  define  the  task  without  specifically  attempting  to 
bring  the  task  analyst  up  to  speed.  Means,  for  example,  used  panels  of  experts  who  checked  each 
others'  versions  of  domain  problems  so  that  consensus  could  be  reached  on  which  sorts  problems 
were  useful  for  study.  Lesgold  included  on  his  analysis  team  an  expert  with  extensive  experience 
training  novices  (Lesgold  &  Lajoie,  1991). 

Developing  focus  problems 

After  defining  the  range  of  problems,  the  analyst  needs  a  forum  for  examining  which 
cognitive  skills  are  brought  to  bear  on  the  problems.  Rather  than  analyze  a  live  task  in  an 
"online"  work  setting,  a  typical  technique  is  to  design  focus  problems  that  contain  critical  features 
of  the  kinds  of  problems  faced  by  people  performing  the  task.  Such  problems  may  be  critical 
incidents,  or  they  may  be  generally  representative  of  the  job  task,  but  they  must  be  entirely 
domain  specific  in  order  to  stimulate  the  metacognitive  skills  and  representations  of  the  domain 
that  are  the  focus  of  the  task  analysis.  By  studying  the  practitioner's  approach  to  solving  these 
problems,  the  analyst  gathers  information  about  the  key  skills  generally  required  to  carry  out  the 
task. 

Roth  and  his  colleagues  (Roth  &  Woods,  1989;  Roth,  Woods  &  Pople,  1992;  Moray 
&Rotenburg,  1989)  convened  panels  of  experts  to  construct  the  focus  problem,  an  approach 
similar  to  the  one  described  by  Means.  After  convening  her  experts  to  define  typical  problem 
types,  Means  (1993)  had  her  experts  construct  problem  examples.  In  their  work  on  avionics 
troubleshooting  systems,  Lesgold  and  his  colleagues  relied  on  their  expert  to  define  the  focus 
problem,  but  his  decision  was  informed  by  protocol  data  (see  below)  from  previous  observations  of 
airmen  working  in  the  problem  area  (Lesgold  et  al.,  1990).  ' 
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Knowledge  elicitation  techniques 

Performance  components.  With  the  focus  task  in  place,  the  analyst  collects  observational  and 
interview  data  concerning  task  performance.  The  data  of  interest  may  occur  at  the  performance 
component  level  as  described  by  Gardner.  For  example,  Lesgold  and  Lajoie  ( 1 99 1 )  measured 
semantic  retrieval  efficiency  on  a  subtask  in  an  effort  to  determine  whether  there  were  differences  in 
the  accessibility  and  organization  of  key  concepts  between  effective  and  less  effective  beginning 
troubleshooters.  Moray  and  Rotenberg  (1989)  collected  eye-movement  data  as  experts  and  novices 
worked  on  a  computer  simulation  of  a  temperature  control  problem  with  relevance  to  power 
stations.  From  this  data  they  formed  hypotheses  about  the  processing  requirements  of  subsets  of 
the  task.  The  data  allowed  them  to  identify  points  at  which  "cognitive  lockup"  occurred.  In  this 
condition  practitioners  fixate  on  one  aspect  of  the  problem  to  the  exclusion  of  other  cues 
suggesting  alternative  solutions. 

Conceptual  knowledge.  An  inevitable  goal  of  any  task  analysis  is  defining  pertinent  domain 
knowledge  (Gardner's  knowledge  structures),  particularly  in  areas  where  procedures  have  become 
automatic.  This  is  largely  accomplished  through  interviews.  Although  there  are  numerous  ways 
to  help  people  retrieve  information  from  memory,  only  the  most  basic  and  important  technique, 
prompting  and  probing,  will  be  mentioned  here  by  way  of  example.  See  Table  1  for  a  summary  of 
many  elicitation  techniques. 

Table  1 

Cordingley's  Knowledge  Elicitation  Techniques 


Technique 


Comments 


Interview 

LaFrance  questions 
Laddering 


All  knowledge  types  will  emerge 
Reveal  schemata  w/6  levels  of  question 
Generate  hierarchies  of  concepts 


Focused  Discussion 
Retrospective  cases 
Forward  scenario 
Critical  incident 
Interesting  cases 
Distinguishing  goals 
Goal  Decomposition 
Decision  Analysis 
Teachback 


All  knowledge  types  will  emerge 
Focus  on  procedural  knowledge 

Often  vividly  remembered 

Reveal  extremes  of  domain  problems 

I.D.  minimum  features  that  define  goal 

Means-end  analysis 

Assign  value  to  each  decision  path 

Doublecheck  gathered  material 


Construct  Elicitation 
Kelly's  diads/triads 
Sorting 


Focus  on  schemata 
Software  automates  elicitation 
Forced  conceptual  categories 


Protocol  Analysis 
Think  aloud 
Eidetic  Reduction 
Retrospective  report 


Focus  on  cognitive  strategies 
Reports  all  thoughts  during  task 
Critical  self-evaluation  of  task 
Memory  of  thinking  process—can  be 
compared  with  real-time  version 


Simulations 


Esp.  useful  if  acquiring  information 
about  system  requirements 


Note.  Table  compiled  from  Cordingley,  (1989). 
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Probes  are  content-free  comments  by  the  interviewer  which  are  meant  to  encourage  the  expert  to 
elaborate  further,  for  example,  "Can  you  tell  me  a  bit  more  about  that?"  Prompts  address  specific 
aspects  of  the  content,  for  example,  "Who  would  you  send  that  to?"  Means  and  Loftus  (1991) 
demonstrated  that  prompts  which  provide  contextual  cues  are  useful  for  decomposing  molar 
(composite)  memories  for  episodic  events.  For  example,  if  the  expert  were  being  asked  to  recall  a 
specific  case  of  an  activity  performed  periodically,  the  analyst  would  prompt  by  asking  the  expert 
to  recall  other  activities  that  occurred  in  the  same  time  period.  Hinkle's  laddering  technique, 
described  by  Cordingley  (1989),  prescribes  a  prompting  method  for  developing  concept  hierarchies. 
The  interviewer  prompts  with  "why"  questions  to  elicit  superordinate  concepts,  "how"  questions  to 
elicit  subordinate  concepts,  and  "other  example"  questions  to  move  laterally  at  the  same  conceptual 
level  in  the  domain. 

Lesgold  took  a  semi-structured  interview  approach  to  analyze  avionics  troubleshooters 
(Lesgold  et  al.,  1990).  He  and  his  colleagues  used  a  means-end  analysis  to  define  the  problem 
space  for  a  focus  system  fault  problem.  Then,  based  on  the  knowledge  of  the  expert,  they  defined 
on  paper  an  "effective  problem  space"  which  consisted  only  of  those  steps  an  expert  would  take 
plus  the  likely  steps  novices  would  take  to  move  through  the  problem  space.  They  developed  a  set 
of  probe  questions  i  n  advance  for  each  step  (or  node)  in  the  effective  problem  space,  then  presented 
the  problem  and  effective  problem  space  on  paper  to  novices.  Lesgold  recorded  the  order  in  which 
novices  worked  through  nodes  of  the  problem  space,  asking  the  probe  questions  for  each  node  as 
the  novice  chose  that  decision  path.  From  the  information  he  gathered,  he  was  able  to  categorize 
deficits  in  novices'  knowledge  base.  He  also  derived  problem-solving  strategies  and  mental  models 
of  the  problem. 

Metacognitive  skills.  To  focus  the  analysis  on  metacognitive  skills,  one  of  the  basic 
elicitation  techniques  is  protocol  analysis  (Ericsson  &Simon,  1985).  In  protocol  analysis,  the 
researcher  records  all  the  actions  and  verbalizations  of  the  expert  as  he  performs  the  task.  From 
this  data  the  analyst  infers  the  expert's  mental  model  of  the  task.  In  verbal  protocol  analysis,  the 
analyst  typically  asks  the  problem  solver  to  "think  aloud"  as  they  complete  a  task.  They  may 
audio  or  video  tape  the  task  and  have  the  problem  solver  retroactively  describe  what  he  was 
thinking  or  doing  at  each  step  as  they  review  the  tape  together.  Means  used  this  method  widely  to 
identify  the  skills  and  knowledge  of  troubleshooters  and  air  traffic  controllers.  In  the  weakest 
version  of  the  method,  the  analyst  simply  asks  the  practitioner  to  describe  post  hoc  what  the  he 
was  thinking  as  he  completed  the  task.  Other  metacognitive  elicitation  techniques 

include  reconstruction  and  asking  practitioners  to  literally  reproduce  their  mental  representations  by 
drawing  them. 

The  reconstruction  technique  (Roth  &  Woods,  1989)  externalizes  practitioners'  conceptual 
frameworks.  The  analyst  briefly  presents  a  picture  or  description  of  a  domain  situation  to  the 
practitioner.  The  practitioner  is  then  asked  to  reconstruct  the  situation  from  memory.  Memory 
distortions  indicate  the  nature  of  the  practitioner's  schemata. 

Examples  of  eliciting  mental  representations  are  provided  by  Means  (1993).  She  asked 
avionics  electronics  technicians  to  draw  their  mental  representations  of  a  test  station  as  they  tried 
to  diagnose  a  fault.  This  information  helped  her  identify  salient  features  of  a  complex  problem 
representation.  She  drew  conclusions  about  air  traffic  controllers'  problem  representations  and 
planning  strategies  by  having  them  draw  the  airspace  post  hoc  as  it  looked  when  they  completed 
the  problem.  Woods  and  Hollnagel  (1988)  also  used  drawings  of  experts'  functional 
representations  of  their  equipment  as  part  of  a  CTA  of  nuclear  power  plant  operators. 

Computers  as  a  tool  for  knowledge  elicitation.  As  a  final  note  about  elicitation  techniques, 
comparing  human  and  computer  problem-solving  performance  can  sometimes  reveal  human 
strategies.  Roth,  Woods,  and  Pople  (1992)  compared  human  performance  in  a  simulation  of  an 
emergency  situation  at  a  nuclear  power  plant  to  computer  performance  in  the  same  situation.  Two 
teams  of  experts  worked  the  problem  while  the  researchers  collected  protocol  and  observational 
data.  By  comparing  the  experts  strategies  to  the  computer's  superior  problem  solution,  Roth  and 
his  team  were  able  to  pinpoint  the  areas  of  the  problem  space  where  the  experts'  schemata 
interfered  with  optimal  problem  solution. 
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Results  of  CTA 

Having  performed  a  CTA,  how  can  one  organize  the  data  in  a  useful  form?  The  task  analyses 
performed  by  Means,  Lesgold,  and  Woods  and  their  colleagues  each  resulted  in  a  wealth  of  data 
organized  in  different  forms. 

From  her  protocol  analyses  Means  (1993)  created  tables  which  related  steps  in  the  problem 
solution  both  to  the  specific  knowledge  which  supported  each  step  and  to  more  general  categories 
which  describe  the  overall  knowledge  base  required  to  do  the  job.  Organized  one  way  the  data 
show  the  specific  knowledge  needed  to  do  a  task.  Organized  another  way  the  data  show  which 
knowledge  is  common  to  various  tasks.  Means  also  tried  to  examine  the  working  mental 
representations  of  her  two  groups  of  practitioners.  Although  shs  never  specifies  how  she  uses  the 
troubleshooter's  mental  representations  in  her  analysis,  she  used  the  air  traffic  controllers' 
representations  to  draw  conclusions  about  using  radar  during  training. 

Woods'  project  comparing  nuclear  power  plant  operators'  problem-solving  to  an  expert  system 
solution  of  the  same  problem  was  an  interesting  sidelight  which  arose  after  the  expert  system  was 
well  developed.  Woods  and  his  colleagues  performed  many  CTA's  during  the  development  of  the 
system,  collecting  a  large  body  of  declarative  and  procedural  knowledge  as  well  as  experts' 
schematic  represent," dons  of  the  physical  systems  with  which  they  work.  To  define  competency 
models  of  task  performance,  they  combined  these  schematic  images  with  text  describing  the 
knowledge  base.  Thus  experts'  pictorial  representation  of  their  mental  models  were  used  as  an 
anchor  to  define  the  important  goal  states  within  the  task.  Performance  models  were  then  derived 
by  posing  problems  that  could  arise  within  the  competency  model  and  documenting  practitioners' 
means-end  solutions. 

In  his  CTAs  of  avionics  troubleshooters  Lesgold  and  his  colleagues  categorized  the  resulting 
protocol  and  interview  data  in  primarily  two  ways.  They  identified  six  knowledge  areas  (see,  Table 
2)  which  listed  all  the  metacognitive,  declarative,  and  procedural  knowledge  required  to  solve  the 
focus  problems. 

Table  2 

One  of  Lesgold's  Data  Organizers 


Information  Type  Examples 


Plans 

Extend  and  test  a  card;  trace  through  schematic  of  card. 

Hypotheses 

Short  is  caused  by  broken  wire;  ground  missing  from  relay. 

Device  and  system 

Understanding  and  use  of  external  control  panel;  understanding  of 

understanding 

grounds  and  voltage 

Errors 

Getting  pin  numbers  for  a  test  wrong. 

Methods  and  skills 

Ability  to  run  confidence  check  programs;  ability  to  interpret 

diagrams  of  relays,  contacts,  coils. 

Systematicity 

Returns  to  point  where  he  knew  what  was  going  on  when  a  dead  end 

was  encountered;  check  path  from  power  source. 

Note.  Adapted  from  Lesgold,  et  al.,  1990. 


They  then  used  this  framework  as  a  diagnostic  scale  to  compare  the  strengths  and  weaknesses  of 
less  and  more  proficient  troubleshooters.  They  also  superimposed  novice  solution  paths  through 
the  problem  space  onto  expert  solution  paths  to  simplify  the  process  of  identifying  salient 
differences  in  the  means-end  solutions. 
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CTA  Caveats 


The  benefit  of  CTA  is  clearly  its  rich  and  thorough  description  of  task  performance.  However 
it  is  not  without  its  drawbacks.  The  biggest  problem  is  resource  intensity.  Gathering  the  data  for 
a  CTA  will  take  hours  of  the  practitioner's  and  analyst's  time-far  more  than  a  traditional  task 
analysis-and  will  generate  huge  quantities  of  data,  only  some  of  which  will  have  broad 
applicability  to  the  problem.  Analyzing  the  data  is  then  another  time-consuming  effort.  As 
Means  (1 993)  points  out,  the  process  of  codifying  and  categorizing  the  data  is  highly  iterative, 
with  new  conceptual  categories  emerging  as  you  proceed  that  may  require  reviewing  previously 
analyzed  material. 

The  CTA  process  is  fertile  ground  for  bias  and  error.  Roth  and  Woods  (1989)  list  three 
dimensions  that  will  determine  the  validity  of  CTA  results: 

"1 )  the  specificity  of  the  information  being  elicited,  2)  .  .  .  the  fidelity  or  realism  of  the  retrieval 
context,  3)  the  length  of  time  between  when  the  information  was  attended  to  by  the  expert  and 
the  time  he  is  asked  to  report  it"  (p.  255).  If  the  CTA  lacks  in  any  of  these  areas  tremendous 
amounts  of  effort  may  be  uselessly  expended.  In  addition,  complex  problems  will  inevitably  have 
a  variety  of  good  solution  paths,  and  for  any  one  path,  a  variety  of  explanations  (Means,  1993). 
Therefore  the  choice  of  experts  and  solutions  studied  is  another  potential  area  of  bias. 

Finally,  there  is  the  question  of  how  to  best  use  the  important  cognitive  and  metacognitive 
data  arising  from  a  CTA,  an  area  researchers  are  only  beginning  to  understand. 

By  very  definition,  experts'  mental  models  and  strategies  have  been  developed  through  the 
slow  process  of  accumulating  experience  in  their  domain  areas.  It  is  completely  unclear  to  what 
extent  t!iis  process  can  be  circumvented.  Strategies,  schemata,  and  organizational  wets  mat 
e.nerge  directly  from  domain  knowledge-the  powerful  tools  of  expertise  which  are  least  subject  to 
articulation  and  conscious  manipulation  by  the  expert-even  if  identified,  may  not  be  transferable. 
And  if  they  are  imposed  on  learners,  they  may  interfere  in  as  yet  unknown  ways  with  the  process 
of  acquiring  expertise. 

On  the  positive  side,  some  lower-level  cognitive  strategies  can  be  taught.  For  example, 
experts'  domain  specific  strategies  for  planning  and  reflecting  on  the  problem  solving  process  will 
emerge  from  a  CTA.  Thsse  can  be  taught  to  novices  through  modelling  (Brown,  Collins,  & 
Dugiud,  1989;  Collins,  Brown  and  Newman,  1990).  Conceptual  models  such  as  a  picture  of  an 
avionics  test-station  focusing  only  on  certain  pieces  of  equipment  are  another  CTA  result.  Similar 
models  which  highlight  the  key  features  of  a  domain  area  have  also  been  shown  to  aid  learning 
(Mayer,  !9§9)  for  lower  ability  students.  Although  in  Mayer's  work  the  conceptual  models  used 
were  not  explicitly  (he  working  models  of  experts,  it  seems  plausible  that  in  some  cases  experts' 
models  could  be  used  productively  in  instruction. 


Implications  of  CTA  for  Instructional  Design 

Similarities 

A  difficulty  for  instructional  designers  is  that  CTA  procedures  were  developed  by  cognitive 
psychologists  without  inference  to  task  analysis  as  described  in  the  instructional  design  (ID) 
literature.  The  many  similarities  between  the  two  approaches  are  obscured  by  differences  in 
terminology  and  emphasis. 

Tc  note  the  most  important  similarity  first,  both  approaches  result  in  a  hierarchical  or 
proceds.'ralized  description  of  task.  None  of  the  articles  reviewed  here  clearly  laid  out  the 
proceduralized  version  of  the  knowledge  uncovered  by  CTA.  However,  all  of  the  CTA's  were 
undertaken  in  the  sen  ice  of  either  expert  system  or  intelligent  tutor  design.  Therefore  to  be  useful, 
the  knowledge  ultimately  had  to  be  put  in  a  rule-based  form  manipulable  by  a  computer  program. 

None  of  the  knowledge  uncovered  by  CTA  is  outside  the  range  of  knowledge  types  described 
by  Gagrie  (cf.  Gsgne,  Briggs,  and  Wager,  1992),  whose  instructional  theory  is  perhaps  the  most 
influential  outside  the  ID  literature  CKyllonen  &  Shute,  1 989).  Because  cognitive  psychologists 
tend  to  work  from  the  perspective  of  Anderson's  ACT*  theory  (Anderson,  1987),  they  use  fewer 
firms  to  describe  the  same  knowledge  types.  But  an  instructional  designer  could  cany  out  a  CTA 
and  describe  the  results  in  Gagne's  terms-verbal  information,  discriminations,  defined  and  concrete 
concepts,  rules,  and  cognitive  strategies.  For  example,  Means'  instantiated  skill  of  avionics 
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troubleshooters:  "Knowing  how  P/S  regulator  is  related  to  power  fail  indicator  lights,"  (Means, 
1993)  can  be  recast  as  a  defined  concept:  "Identify  characteristics  of  faulty  P/S  regulator."  Paths 
through  a  problem  space,  including  planning  and  reflective  steps,  can  easily  be  recast  as  a  set  of 
rules. 

A.  finai  similarity  worth  considering  is  that  between  Gagne's  learning  hierarchies  and  Wood's 
competency  model.  A  learning  hierarchy,  with  its  emphasis  on  what  is  to  be  achieved,  not  how, 
is  conceptually  similar  to  a  competency  model.  However,  where  Gagne's  hierarchy  lists  all  the 
actions  that  lead  to  the  desired  goal-state,  Wood's  competency  model  lists  each  object  that  can  be 
acted  on  and  the  actions  that  can  be  performed  on  that  object  in  the  service  of  the  desired  goal-state. 

Barriers  to  Integrated  Models 

Given  the  above  discussion,  a  question  that  must  then  be  asked  is,  "Should  cognitive 
psychologists  conducting  CTAs  as  part  of  an  instructional  design  process  be  referring  to  the  ID 
literature?"  The  answer,  unfortunately,  appears  to  be  "Why  bother?" 

Perhaps  most  damaging,  the  ID  literature  is  weak  when  it  comes  to  addressing  the  importance 
of  metacognitive  skills  in  problem  solving.  Despite  uncertainty  about  how  to  incorporate  such 
skills  in  instruction,  they  have  emerged  as  crucial  factors  in  expert  performance  and  are  some  of  the 
richest  data  obtainable  through  a  CTA.  While  the  importance  of  domain-specific  metacognitive 
skills  is  acknowledged  by  Gagne  et  al.  ( 1992),  they  also  believe  that  on  a  cost-benefit  basis,  ID 
resources  are  better  focussed  on  the  verbal  information  and  intellectual  skills  (rules  and  concepts) 
within  a  domain  (see,  Gagne  et  al.,  1992,  p.  72).   A  look  at  Dick  and  Cary  (1985)  reveals  no 
mention  of  metacognition  or  cognitive  strategies  in  any  form. 

Secondly,  cognitive  psychologists  have  designed  instruction,  based  on  the  results  of  CTA's,  in 
which  they  provide  successful  remediation  defined  only  in  terms  of  declarative  and  procedural 
knowledge.  Most  notable  is  Lesgold's  intelligent  tutor,  Sherlock  (see,  Lesgold  &  Lajoie,  1992, 
for  an  overview).  Sherlock  simultaneously  addresses  gaps  in  novices  domain  knowledge,  domain- 
specific  problem-solving  cognitive  strategies,  and  generic  problem-solving  skills  in  a 
contextualized  form. 

Finally,  cognitive  psychologists  have  defined  a  taxonomy  of  learning  outcomes  which  better 
reflects  the  knowledge  structures  that  have  emeiged  from  cognitive  psychological  research 
(Kylionen  &  Shute,  1989).  This  taxonomy  defines  propositions  as  the  simplest  knowledge 
structure  and  mental  models  as  the  most  complicated.  The  same  research  effort  which  applied 
CTA's  to  avionics  troubleshooters  and  air  traffic  controllers  is  now  poised  to  test  this  taxonomy 
and  to  systematically  investigate  instructional  strategies  based  on  it. 

The  opposite  question  might  well  be  asked:  "Could  instructional  designers  benefit  from  a 
CTA  approach  to  task  analysis?"  Here  an  affirmative  answer  seems  appropriate.  Most  examples 
of  task  analysis  in  ID  how-to  texts  are  of  tasks  where  the  designer  qualifies  as  an  expert: 
introspection,  simple  research,  or  naturalistic  observation  are  all  that  are  required  for  the  analysis. 
However,  in  the  real  world  instructional  designers  frequently  rely  on  subject-matter  experts  for  task 
descriptions.  Merely  viewed  at  the  most  technical  level  as  a  collection  of  methods  for  working 
with  subject-matter  experts,  CTA  still  adds  a  valuable  dimension  of  specificity  to  ID's  usually 
vague  prescriptions  for  task  analysis 

Conclusion 

•     Having  traced  the  roots  of  CTA,  reviewed  its  methods,  and  examined  some  examples  in 
practice,  it  is  obvious  that  researchers  outside  the  field  of  ID  are  making  major  strides  in  the 
important  area  of  analyzing  real-world  problem  solving  tasks.    If  ID  is  to  grow,  or  even  survive, 
as  an  interesting  academic  pursuit,  it  must  Find  some  way  to  incorporate  and  make  use  of  this  and 
other  contributions  from  outside  disciplines.  At  the  very  least,  ID  texts  could  incorporate  CTA 
into  the  systematic  design  of  instruction.  It  could  be  cast  as  a  task  analysis  subroutine  to  be 
followed  in  the  case  of  problems  meeting  the  criteria  by  Funke  which  were  presented  at  the 
beginning  of  this  paper.  Gagne  et  al.  (1992)  already  differentiate  the  design  procedure  for 
intellectual  skills  from  those  for  motor  or  affective  skills. 

However  this  solution  begs  the  question  to  some  extent.  At  issue  are  fundamental  theoretical 
principles  of  learning.  Cognitive  task  analysis  and  traditional  task  analysis,  with  their  emphases 
on  the  practitioner  and  the  task  respectively,  reflect  the  theoretical  divergence  between  behaviorism 
and  constructivism.  As  cognitive  psychologists  focus  their  research  efforts  on  identifying  and 
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instructionally  manipulating  complex  mental  models,  they  will  continue  to  make  headway  in 
defining  instructional  techniques  for  teaching  in  complex  problem-solving  domains.  Unless  ID 
also  finds  a  way  to  capitalize  on  the  importance  of  cognition  in  mediating  task  performance,  it  will 
not  have  the  theoretical  power  to  handle  such  tasks. 


Footnotes 

1.  In  general,  it  is  a  poor  strategy  to  accept  the  task  descriptions  and  solutions  of  one  expert  as 
a  basis  for  examining  the  cognitive  skills  applied  by  others  on  the  task,  as  Lesgold  and  Lajoie 
(1991 )  acknowledge.  In  doing  so  the  analyst  runs  the  risk  of  missing  critical  aspects  of  the  task 

not  articulated  by  their  particular  expert. 
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Lessons  in  Communication:    Three  Contract  Ethnographies 

Abstract 

Ethnography,  the  preferred  method  of  inquiry  of  the  cultural  anthropologist,  is  currently  the  most  widely 
recognized  qualitative  method  for  research.  Defined  as  the  "art  and  science  of  describing  a  group  or  culture" 
(Fetterman,  1989,  p.  11),  it  provides  a  means  of  studying  and  understanding  other  groups,  their  values  and 
culture.  This  study  documents  the  use  of  a  permutation  of  that  method,  called  "contract  ethnography," 
where  a  trained  professional  enters  a  culture  for  a  limited  period  of  time  with  a  limited  set  of  questions.  In 
the  case  of  this  study,  three  programs  funded  by  the  Center  for  Substance  Abuse  Prevention  (CSAP)  are 
examined:  (1)  the  Creating  Lasting  Connections  (CLC)  Project  based  upon  church  involvement  in 
Louisville,  Kentucky;  (2)  the  Beyond  Blue  Bay  (BBB)  Project  located  on  an  Indian  reservation  in  Ronan, 
Montana;  and  (3)  the  Child  Development  Project  (CDP)  based  in  public  elementary  schools  headquartered  in 
Oakland,  California  (with  a  second  site  visit  to  Louisville,  Kentucky).  Each  of  the  two-person  site-visits 
occurred  during  the  third  of  five  years  of  federal  funding.  One  team  member  was  a  a  senior  staff  member  of 
the  Washington,  D.C.-based  consulting  firm,  CSR,  Inc.,  termed  the  "Site  Evaluator,"  and  the  second,  a 
person  trained  in  techniques  of  contract  ethnography,  called  the  "Site  Visitor."  Two  parallel  reports  were 
written  for  each  project,  focusing  upon  different  aspects  of  the  project.  The  first  author  of  this  paper  served 
as  the  Site  Visitor  on  each  of  these  three  projects,  attempting  to  document  aspects  of  the  local  culture  and 
its  relation  to  the  operation  of  the  specific  program.  Because  each  of  these  projects  had  developed  its  own 
6-8  minute  promotional  videotape,  this  paper  examines  communication  lessons  that  can  be  drawn  from 
these  video  presentations. 


The  Method 

Ethnography  has  enjoyed  wide  acceptance  as  a  qualitative  method  of  inquiry.  With  expositions  provided  by 
Clifford  Geertz  (1973),  David  Fetterman  (1988,  1989)  and  others,  the  method  has  been  widely  adopted  as  a 
methodology  for  study  in  education,  health,  and  other  social  science  areas.  In  many  ways,  ethnography  can 
be  considered  the  "grandfather"  of  other  qualitative  methods  in  use.  In  fact,  one  problem  of  qualitative 
methods  today  is  that  for  many  people,  "qualitative  inquiry"  is  synomymous  with  "ethnography"(Lancy, 
1993). 

The  strengths  of  ethnography  come  from  its  grounding  in  extensive  observation  over  long  periods  of  time, 
its  adoption  of  the  "insider's"  (emic)  perspective,  its  "value  neutrality,"  and  its  ability  to  detect  fine 
differences  in  culture.  A  major  difficulty  with  ethnography  is  the  relatively  long  periods  of  time  required  on 
location.  While  such  grounding  in  the  culture  may  be  necessary  to  write  a  book-length  definitive  study,  as 
this  methodology  has  been  employed  by  U.S.  government  agencies,  the  extensive  lengths  of  time  required 
have  entailed  expenses  that  are  prohibitive.  Deslonde  (1981)  reports  on  a  nationwide  study  where  full-time 
data  collectors  used  ethnographic  techniques  to  study  the  multiple  sites  of  a  national  project  for  a  full  year. 
Eastmond,  one  of  the  authors  of  this  paper,  has  been  employed  one-fourth  time  over  the  past  4  years  as  an 
ethnographer-in-residence  to  study  a  federally  funded  experiment  in  child  development  (the  Comprehensive 
Child  Development  Project),  at  one  of  36  implementation  sites  nationwide.  To  date,  however,  not  many 
projects  have  been  willing  to  devote  that  level  of  resources  to  ethnographic  inquiry.  Clearly,  given  current 
funding  trends,  there  is  a  need  for  a  method  that  allows  for  the  same  kind  of  qualitative  insights  without  the 
huge  investment  in  time  and  effort  required  by  full  ethnography. 

A  promising  development  for  many  situations  has  been  entitled  "contract  ethnography"  and  involves  a  more 
focused  effort  over  a  shorter  period  of  time.  This  kind  of  inquiry  goes  under  several  names,  two  others 
being  "case  study  research"  and  "microethnography."  For  this  paper,  the  approach  will  be  termed,  "contract 
ethnography."  An  important  feature  of  this  form  of  study,  as  the  name  implies,  is  the  use  of  contractual 
arrangement  in  designing  and  conducting  the  study.  The  contractual  arrangement  calls  for  examination  of  a 
limited  set  of  questions,  applied  to  a  specific  prcrarn.  While  retaining  many  of  the  techniques  and 
assumptions  of  traditional  ethnography,  this  approach  allows  more  rapid  entry  and  exit  from  a  particular 
project,  building  upon  the  notions  of  expertise  espoused  by  Eisner  (1991). 
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For  this  paper,  the  contract  ethnographer  was  Nick  Eastmond.  When  references  to  "I"  or  "we"  are  made,  it 
is  to  Eastmond  or  to  the  particular  site  visit  team.  Other  terminology  used  is  that  in  current  use  at  the  sites 
visited,  for  example,  both  "Native  American"  and  "Indian"  are  used  by  various  people  at  the  Montana  site. 

Contract  Ethnography  for  CSAP 

The  studies  reported  here  were  part  of  a  national  cross-site  evaluation  conducted  for  the  Center  for  Substance 
Abuse  Prevention  (CSAP)  of  90  selected  high  risk  youth  (HRY)  demonstration  grants  funded  in  fiscal  years 
1989,  1990,  and  1991  in  different  parts  of  the  United  States.  Each  of  these  projects  targetted  high  risk 
youth  (HRY)  in  attempting  to  prevent  their  abuse  of  alcohol  and  other  drugs  (AOD).  The  purpose  of  the 
site  visits  and  subsequent  reports  was  to  "understand  each  grantee's  basic  assumptions  in  selecting  and 
organizing  its  interventions,  to  determine  how  the  program  operates,  and  to  reveal  factors  that  are  viewed  as 
critical  to  successful  outcomes."  Initially,  the  inquiry  sought  to  identify  10-12  highly  successful 
"lighthouse"  projects,  but  after  the  first  year  this  expectation  was  dropped,  presumably  due  to  unhealthy 
competitive  pressures  that  resulted  among  projects. 

The  approach  required  that  two  outsiders  visit  ongoing  projects  during  their  third  year  of  a  five-year  funding 
cycle.  The  two  outsiders,  one  called  the  "Site  Visitor"  (Eastmond  in  this  report)  and  the  other  the  "Site 
Evaluator",  spent  up  to  10  days  visiting  the  site.  The  "site  visitor"  had  responsibility  for  reporting  the 
local  culture  —  issues,  trends,  and  values  --  that  impinge  upon  the  workings  of  the  prevention  project,  while 
the  "site  evaluator"  had  responsibility  for  documenting  the  third  party  evaluation  effort. 

In  contract  ethnography,  the  investigator  (here  the  "Site  Visitor")  spends  a  limited  amount  of  time  on  site, 
usually  one  week  to  ten  days  in  the  CSAP  projects.  A  set  of  general  questions  are  provided  by  the  funding 
agency  to  guide  the  inquiry.  Typical  questions  would  be: 

•  What  are  the  magnitude,  distribution,  and  character  of  the  target  groups  [for  the  program]  in  the 
community? 

•  What  are  the  values,  culture,  and  norms  of  the  program's  target  population?  What  implications 
do  these  values,  culture,  and  norms  have  on  the  High  Risk  Youth  program  feasibility  and  target  group 
participation? 

•  What  are  the  major  political  and  social  issues  in  the  community,  and  how  have  these  issues 
affected  the  services  and  support  networks  existing  prior  to  the  High  Risk  Youth  program? 

(CSR,  Inc.,  1992). 

It  should  be  noted  that  investigating  the  questions  above  could  provide  the  basis  for  one  or  more  books  on 
the  subject;  however,  to  be  useful,  a  more  truncated  response  was  expected.  While  members  of  a 
community  might  be  aware  of  hundreds  of  local  issues,  the  Site  Visitor  might  examine  only  three  or  four, 
but  those  would  be  the  three  or  four  seen  as  most  crucial  and  most  relevant  to  the  project  at  hand. 
Although  the  people  on  site  frequently  suggested  the  issues  they  felt  were  crucial,  the  final  selection  of 
issues  to  highlight  necessarily  remained  with  the  Site  Visitor,  as  did  editorial  control  of  the  written 
document. 

Part  of  the  job  requirement  for  this  kind  of  work  is  that  only  a  specific  amount  of  time  is  alloted  to  do  the 
work.  For  the  three  reports  examined  here,  the  allotted  number  of  person-days  was  fourteen,  with  one  week 
to  10  days  on  site  and  the  remainder  in  analyzing  and  writing  up  the  results.  Where  information  was 
missing,  it  could  be  supplemented  by  later  followup  telephone  interviews  to  the  site.  The  three  reports 
considered  here  were  extensive,  with  the  number  of  typewritten  pages  varying  from  27  in  the  edited  version 
for  the  CLC  Project  to  74  pages  (unedited)  in  the  CDP. 

These  studies  necessarily  rely  on  the  basic  assumptions  of  qualitative  research  —  namely,  that  the  researcher 
is  by  definition  the  best  means  of  gathering  information,  and  that  judgment  and  expertise  must  be  exercised 
in  doing  so.  In  the  last  analysis,  however,  the  report  is  the  work  of  a  single  researcher  and  is  colored  by  the 
background  and  perspective  of  that  person.  The  contract  ethnographer  attempts  to  be  candid  about 
observations,  but  also  to  be  fair  in  assessing  all  factors,  motives  and  constraints  operating  on  and  within  a 
program.  ] 
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In  gathering  data,  these  studies  make  use  of  the  qualitative  research  techniques  employed  by  ethnography, 
including  participant  observation,  interviews  with  persons  both  inside  and  outside  of  the  program,  analysis 
of  grantee  records  and  reports,  and  a  sampling  of  local  media  sources.  Questions  arising  in  the  earlier  stages 
of  the  research  are  refined  and,  where  necessary,  revisited. 

One  final  consideration  is  that  here  are  times  in  contract  ethnography  when  the  questions  posed  by  the 
funding  agency  seem  less  relevant  at  the  particular  site  than  they  might  be  at  others.  In  the  experience 
doing  the  CSAP  contracts,  when  a  conflict  arose  between  the  prepared  questions  and  the  contract 
ethnographer's  instincts  on-site  of  what  was  important,  it  was  always  possible  to  pursue  the  researcher's 
questions.  While  the  funding  agency  might  fault  a  report  on  omitting  an  answer  to  a  question  (and  simply 
saying  "Findings  indeterminate"  or  "This  answer  is  covered  in  Section  X  in  the  report"  would  be  sufficient), 
in  no  case  did  the  funding  agency  raise  questions  as  to  the  relevance  of  the  findings  provided. 

When  the  report  did  get  back  to  the  grantee,  after  extensive  editing  by  the  consulting  firm  and  perusal  by  the 
CSAP  project  officer  in  Washington,  there  could  be  matters  of  fact  or  interpretation  that  would  be 
negotiated  between  the  Site  Visitor  and  the  grantee.  The  procedures  for  "member  checking"  advocated  by 
Lincoln  and  Guba  (1985),  where  stakeholders  conduct  a  preliminary  reading  of  the  draft  and  negotiate  the 
acceptable  version  with  the  qualitative  researcher,  were  not  acceptable  to  the  funding  agency  and  were  not 
followed  here.  Because  continued  funding  did  not  depend  upon  anything  in  either  the  Site  Visitor  or  the 
Site  Evaluator  reports,  and  because  the  project  officers  took  the  stance  that  they  wanted  the  unedited  account 
of  site  visitors  straight  from  the  field,  the  kind  of  negotiation  with  the  grantees  over  report  content  was 
curtailed. 

In  no  instance  did  the  kind  of  conflict  reported  by  Bell  and  Jankowiak  (1992)  surface  in  this  work.  These 
anthropologists  report  a  persistent  difficulty  in  contract  ethnography  work  when  a  local  person,  perceived  as 
an  expert  by  the  community,  clashes  in  interpretation  with  the  visiting  contract  ethnographer  ("the  folk 
expert  vs.  the  ethnographer).  A  reason  for  this  lack  of  conflict  may  have  been  the  mandate  provided  by  the 
funding  agency,  putting  the  outsiders  in  a  position  of  less  vulnerability,  combined  with  the  reporting 
requirements  that  introduced  a  time  lag  between  the  site  visit  and  the  grantees  receiving  the  report.  Because 
the  reports  have  not  yet  been  made  public,  to  say  nothing  of  being  widely  disseminated,  there  has  been  little 
conflict  to  date  over  findings.  McKenna  (1994)  has  reported  on  the  kinds  of  role  conflicts  that  this  kind  of 
work  poses  for  persons  trained  in  traditional  ethnography. 

Reports  on  the  projects  used  a  variety  of  methods  of  analysis  and  reporting,  to  include  (1)  extensive  use  of 
photographs  taken  on  site  and  in  the  local  community;  (2)  maps  of  the  larger  geographic  area  and  of  the 
particular  project  sites;  (3)  cartoons,  news  clippings,  sometimes  drawn  from  the  project's  own  "clip  file", 
what  Fetterman  calls  "outcroppings";  and  (4)  charts  and  diagrams  similar  to  those  advocated  in  Miles  & 
Huberman,  1994.  Examples  of  the  latter  are  shown  as  Figures  1  and  2  of  this  paper,  patterned  after  the 
"Case  Dynamics  Matrix"  (pp.  148-151).  These  figures  provide  a  more  efficient  means  of  comparing  the 
three  projects  under  consideration. 

Introductory  videotapes.  The  contents  of  the  project  videos  will  be  described  in  the  sections  that 
follow,  but  at  this  point  it  is  necessary  to  state  how  and  when  they  were  encountered,  since  that  event  varied 
for  each  project.  For  the  first  project,  the  CLC  in  Kentucky,  the  video  did  not  exist  at  the  time  of  the  site 
visit  in  1992.  It  was  a  bit  of  additional  information  gathered  two  years  later  at  the  time  of  the  site  visit  to 
the  Jefferson  County-based  CDP  Project,  also  located  in  Louisville.  Consequently,  the  CLC  video,  unlike 
the  other  two,  was  only  viewed  after-the-fact,  after  the  Site  Visitor  report  had  been  written.  The  video  for 
the  BBB  project  in  Montana  was  first  shown  to  the  site  visit  team  during  a  staff  meeting  on  the  second  day 
of  the  visit.  Because  the  video  was  an  artifact  that  could  be  taken  away  and  examined  later,  it  became  an 
important  source  consulted  in  completing  the  Site  Visitor  report.  The  CDP  video  was  first  mentioned  in 
conversation  during  the  site  visit  in  Louisville,  flippantly  as  "Our  six  minute  video  that  really  takes  8 
minutes."  However,  it  was  not  something  that  we  actually  viewed  until  our  visit  2  weeks  later  to  the  DSC 
headquarters  in  Oakland,  California.  At  that  time,  we  watched  it  with  the  project's  Research  Director  as  an 
introduction  to  the  concepts  behind  the  project.  Again,  the  video  was  an  artifact  that  could  be  taken  from 
the  site,  and  it  was  a  source  reviewed  in  completing  the  Site  Visitor  report. 
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In  the  analysis  of  the  project  videos  described  in  the  remainder  of  this  paper,  the  key  issues  were  identified 
by  three  people:  the  Site  Visitor  (Nick  Eastmond),  a  graduate  assistant  for  qualitative  research  (Tamara 
Walser),  and  a  Native  American  co-presenter  and  graduate  student  in  Instructional  Technology  (Reba  Teran). 
While  watching  the  project  videos,  each  person  identified  issues  individually,  but  these  were  discussed 
collectively  afterward  to  determine  2-3  ones  to  highlight.  This  event  occurred  well  after  the  writing  of  the 
three  Site  Visitor  reports. 

The  Projects 

CLC  Project:     Louisville,  Kentucky 

The  first  project  studied  was  the  Creating  Lasting  Connections  (CLC)  Project  in  Louisville,  Kentucky, 
with  the  site  visit  taking  place  from  July  25-31,  1992.  CLC  is  a  project  undertaken  by  COPES,  Inc. 
(Council  on  Prevention  and  Education:  Substances,  Inc.),  a  nonprofit  organization  providing  alcohol  and 
other  drug  (AOD)  prevention  services  in  the  Louisville  area  for  over  a  decade.  Working  through  the 
congregations  of  local  Protestant  and  Catholic  churches  in  both  the  African  American  and  White 
communities,  the  project  set  up  recruitment  teams  and  later  training  sessions  for  both  parents  (24  weeks) 
and  teenage  youth  (8  weeks).  The  sessions  focused  upon  parenting  skills  and  improved  communications 
and  culminated  in  the  convening  of  joint  activities  where  sharing  between  parents  and  their  teenagers  could 
take  place. 

As  outsiders,  the  site  visit  team  attended  training  sessions  at  both  rural  and  urban  sites,  to  include  a 
planning  meeting  of  the  congregational  team  responsible  for  implementing  training  in  the  inner  city  area  of 
the  Louisville  West  End.  The  team  interviewed  youth,  parents,  staff  and  advisory  council  members,  as  well 
as  access  project  records  and  research  reports.  Important  issues  summarized  in  Figure  1  are  described  in 
more  detail  as  follows: 

(1)  Safety  net  philosophy:  The  project  believed  in  building  positive  communication 
experiences  between  parents  and  their  teens.  However,  given  the  possibility  of  opening  up  major  conflicts 
and  then  lacking  the  resources  to  handle  them,  the  CLC  project  had  retained  the  part-time  services  of  a 
counseling  psychologist.  Any  workshop  participant  failing  to  attend  a  session  would  receive  a  follow-up 
phone  call  to  insure  that  any  lingering  problems  could  be  dealt  with  through  counseling.  The  motto  was: 
"We  don't  pick  scabs,"  i.e.  we  don't  open  past  wounds  and  then  lack  the  means  to  remedy  the  situation. 

(2)  Strong  leader:  The  project's  Executive  Director,  Mr.  Ted  Strader,  was  well  known  in  drug 
abuse  prevention  circles  as  a  person  with  strong  ideas  and  plans  to  carry  them  out.  During  the  first  day  on 
site,  this  model  for  strong  leadership  was  apparent.  The  project,  in  fact,  had  served  as  a  training  ground  for 
prevention  professionals,  studying  under  Mr.  Strader  and  then  being  hired  off  by  other  agencies.  The  down 
side  of  strong  leadership  was  mat  some  people  reacted  personally  against  this  approach,  and  so  there  were 
some  antagonistic  forces  in  the  community  to  be  reckoned  with.  People  involved  in  the  program,  however, 
developed  strong  feelings  of  loyalty  both  to  Mr.  Strader  and  to  the  project. 

(3)  Resiliency  factors  of  church  attendee:  Because  this  project  drew  upon  members  of 
existing  church  congregations  and  because  involvement  with  organized  religion  is  considered  as  one 
"strengthening  factor"  in  predicting  probable  involvement  with  alcohol  or  other  drugs,  it  could  be  argued 
that  this  project  was  working  with  people  predisposed  to  succeed.  However,  interviews  with  many 
participants  provided  convincing  evidence  that  the  AOD  struggle  was  a  major  one  for  them  or  for  their  loved 
ones.  The  CLC  Project  staff  members'  argument  was  thai  ..«ginning  with  one  factor  in  their  favor  meant 
that  families  would  more  likely  stay  through  to  completion  with  the  intense  program  (24  weeks  of  evening 
meetings  -  one  per  week  -  for  parents,  9  weeks  of  similar  meetings  for  their  teenagers,  with  meetings 
lasting  2-3  hours  each  time).  Working  with  the  members  of  the  congregations  seemed  to  provide  an 
effective  way  of  obtaining  and  maintaining  loyalty  of  participants.  Using  this  ingenious  approach  secured 
the  willing  participation  of  congregation  members.  Pastors  and  clergy  of  the  churches  saw  CLC 
involvement  as  a  way  to  prevent  AOD  problems  and  at  the  same  time  build  feelings  of  unity  among 
congregational  members. 

CLC  Project  video:  The  introductory  video  for  the  project  was  produced  and  directed  by  the  Project's 
Executive  Director,  Mr.  Ted  Strader.  Some  of  the  most  convincing  arguments  in  favor  of  the  project  were 
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provided  by  direct  statements  by  Mr.  Strader.  Other  parts  of  the  video  included  actual  footage  from  training 
sessions  involving  parents  sharing  feelings  of  both  love  and  concern  for  their  children.  The  emotional 
impact  of  the  testimonials  is  strong.  Viewers  quickly  perceive  the  strong  commitment  of  participants 
toward  the  value  of  the  project  and  its  positive  impact  on  their  lives. 

The  presentation  may  cause  some  individuals  to  question  whether  they  could  be  that  open  and  forthright 
about  personal  family  problems,  but  the  modeling  from  the  people  pictured  suggests  that  many  people  have 
benefited  from  these  sessions.  Another  concern  is  whether  the  project  will  be  in  existence  long  enough  to 
handle  whatever  problems  were  uncovered.  One  lasting  impression  of  this  site  visitor  could  be  paraphrased 
in  Mr.  Strader's  words  during  an  interview.  "This  project  is  here  to  stay.  We  have  been  here  a  decade,  and 
regardless  of  the  funding  we  receive  from  the  Feds,  we  will  be  here  for  at  least  that  long  in  the  future." 
That  argument  seemed  to  be  a  powerful  one  in  convincing  both  ministers  and  lay  members  that  this  project 
had  a  long-term  commitment  to  AOD  prevention. 


Beyond  Blue  Bay  Project:    Ronan,  Montana 

The  second  site  visit,  November  7-13,  1993,  was  to  the  Beyond  Blue  Bay  (BBB)  Project  on  the  Salish- 
Kootenai  Reservation  in  northern  Montana.  The  current  project  built  upon  the  legacy  of  a  previous 
program  of  substance  abuse  prevention  at  the  Blue  Bay  Healing  Center  on  Flathead  Lake.  In  the  followup 
project  (BBB),  however,  the  program's  emphasis  was  upon  community  development  activities  in  six  of  nine 
reservation  communities,  with  plans  in  place  to  expand  to  the  remaining  3  communities  in  the  final  project 
years.  The  program  attempted  to  strengthen  resilience  factors  based  in  the  traditional  values  of  the  Native 
American  community,  helping  youth  gain  a  more  clear  understanding  of  their  heritage  and  better 
appreciation  of  their  cultural  roots  through  a  variety  of  activities. 

In  addition  to  the  Site  Visitor  and  Site  Evaluator,  a  Navajo  woman,  Trisha  Poleski,  was  part  of  the  site 
visit  team.  Although  she  had  never  visited  this  particular  reservation  before,  her  insights  into  the  problems 
faced  by  Native  Americans  and  her  help  in  sensitizing  the  other  two  outsiders  was  invaluable.  She  was 
frequently  able  to  examine  cultural  practices  on  this  reservation  in  light  of  her  own  experience  with  the 
Navajos  and  suggest  a  new  way  of  looking  at  the  particular  situation.  She  was  also  able  to  make 
suggestions  on  the  draft  report  that  made  substantial  differences  in  tone  and  content. 

BBB  Project  Issues 

Issues  that  surfaced  during  the  visit,  worth  consideration  now  were  as  follows: 

(1)  Community/tribal  divisions.  During  the  early  1900's  the  tribal  lands  of  the  Flathead 
Reservatrion  were  opened  to  homesteading,  with  the  result  that  Caucasians  now  outnumber  Indians  by  a 
ratio  of  3:1  (The  1990  census  lists  21,259  persons  living  in  the  Flathead  Reseervation  area,  and  of  these 
5, 1 30  or  24%  are  listed  as  Indian).  From  a  reading  of  two  local  papers,  the  Lake  County  Leader  in  Ronan, 
with  a  primarily  white  audience  and  the  Char-Koosta  News  in  Pablo,  read  mainly  by  the  Native  American 
community,  it  was  clear  that  community  divisions  exist  and  that  the  points  of  view  of  the  two  groups  are 
often  quite  different.  In  addition  to  those  differences,  the  combining  of  the  two  tribes,  the  Salish  and  the 
Kootenais,  appeared  to  be  a  historical  move  on  the  part  of  the  U.S.  Government.  The  two  groups  have 
different  languages,  different  tribal  customs,  and  often  different  outlooks  on  their  common  issues. 
Designing  a  project  that  could  include  elements  of  both  tribal  groups,  as  well  as  both  Whites  and  Native 
Americans,  which  this  project  has  attempted  to  do,  should  be  considered  a  major  feat.  In  many  ways  visible 
to  us  as  observers,  the  project  was  able  to  benefit  all  groups.  For  example,  community  development  efforts 
were  conducted  in  both  predominately  Salish  and  Kootenai  area.  Programs  like  the  "Soaring  Eagles" 
drumming  groups  for  youth  were  open  to  both  Native  American  and  white  youngsters.  The  survey  of  drug 
and  alcohol  use  among  teenagers,  conducted  by  a  California  consulting  firm  and  funded  by  the  project,  had 
purposely  worked  through  the  public  schools  to  obtain  results  that  would  be  useful  to  administrators  in 
schools  for  both  whites  and  Indians. 

(2)  Fostering  Native  American  culture  in  small  populations.   Both  Salish  and 
Kootenai  tribes  had  their  separate  cultural  centers,  and  elements  of  the  BBB  Project  were  organized  to  foster 
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their  growth.  It  became  clear  that  these  languages,  particularly  Salish,  was  now  spoken  by  only  a  handful 
of  people,  and  that  unless  work  was  done  to  teach  it  to  the  youth,  that  these  languages  would  completely 
die  out.  Because  of  Bureau  of  Indian  Affairs  and  other  government  policies  during  the  1950's,  the  "melting 
pot"  notion,  encouraging  (and  enforcing)  assimilation  into  the  Anglo  culture  at  any  price,  the  price  of 
survival  for  many  was  turning  their  back  on  Indian  ways.  To  combat  alcohol  and  other  drugs,  a  sense  of 
heritage  is  seen  by  these  people  as  a  particularly  effective  factor  in  resiliency.  Learning  how  close  these 
languages  were  to  complete  extinction  put  a  note  of  urgency  in  the  attempts  to  build  language  skills  and 
foster  cultural  appreciation  among  the  youth. 

BBB  project  video:  This  filming  was  completed  by  a  camera  crew  from  the  University  of  Wisconsin, 
while  filming  a  longer  documentary.  Project  staff  used  a  short  segment  of  the  longer  production  as  a  way 
to  represent  their  project  to  outsiders.  Having  outsider  narration  but  tribal  members  testimonials  enhanced 
credibility.  The  emotional  impact  of  the  video  is  strong,  emphasizing  the  devastating  effects  of  alcoholism 
among  the  local  people  and  pointing  to  the  Beyond  Blue  Bay  Project  as  one  ray  of  hope  in  turning  the 
situation  around.  The  project  provides  persuasive  testimonial  of  how  the  cycle  of  negativism  and  shame, 
reinforced  by  alcoholism,  can  be  broken.  Its  emphasis  upon  healing,  through  traditional  Indian  ways, 
suggests  a  positive  basis  for  action. 


The  Child  Development  Project:  Oakland,  California 

The  third  project,  the  Child  Development  Project  (CDP)  of  the  Developmental  Studies  Center  based  in 
Oakland  California,  operated  through  the  elementary  schools  in  six  locations  nationwide:  three  in 
California  and  three  funded  by  CSAP  in  White  Plains,  New  York;  Dade  County,  Florida;  and  Jefferson 
County,  Kentucky.  The  program  trained  teachers  in  three  innovative  practices:  cooperative  learning, 
literature-based  writing,  and  developmental  discipline.  The  project  worked  to  train  teachers  to  use  new  and 
more  democratic  teaching  methods  to  build  responsibility  in  students. 

Our  site  visit  went  to  the  two  schools  implementing  this  model  in  the  Jefferson  County  School  District  in 
Louisville,  Kentucky  (March  12-17,  1994)  and  then  to  the  Developmental  Studies  Center  in  Oakland, 
California'(March  22-25,  1994).  The  visit  to  Jefferson  County  was  timed  to  coincide  with  training  for 
teachers  in  aspects  of  the  CDP  model.  Most  impressive  in  this  training  was  the  way  that  local  teachers, 
designated  as  part  of  the  implementation  team  (I-Team),  were  capable  of  conducting  a  complex  workshop 
for  their  peers. 

The  CDP  was  the  largest  and  most  fully  developed  of  the  projects  visited.  The  staff  in  Oakland  was 
impressive  in  both  their  credentials  and  dedication,  acting  on  a  scale  of  curriculum  reform  that  is  rare  in 
public  education.  The  program  proposed  is  extensive  and  sophisticated.  Most  teachers  interviewed  were 
enthusiastic  about  it,  for  its  prosocial  values  and  its  emphasis  upon  creating  a  "caring  community."  Value; 
of  free  expression,  mutual  respect  for  members  of  diverse  groups  and  community  service  were  evident.  But 
becoming  expert  in  its  implementation  is  not  a  trivial  task;  to  teach  in  this  way  requires  extensive  training 
and  practice. 


Issues: 


Specific  issues  examined  were; 

(1)  Interface  of  the  Project  with  KERA.  The  schools  in  Kentucky  are  undergoing  an 
extensive  program  entitled  the  Kentucky  Educational  Reform  Act  (KERA),  fostering  ungraded  primary  grade 
classes  and  extensive  testing  of  student  progress.  The  program  is  complex,  but  provides  each  school  with 
specific  goals  for  student  achievement  and  a  "carrot  and  stick"  approach  to  make  schools  accountable. 
Schools  meeting  or  exceeding  their  goals  are  given  more  latitude  in  governance  and  are  provided  with  extra 
funding  to  be  used  in  ways  the  teachers  specify.  Schools  not  meeting  their  goals  are  required  to  develop  a 
plan  for  turning  the  achievement  patterns  around,  and  if  continually  unsuccessful,  may  have  new  leadership 
imposed  upon  them  from  the  outside.  Implicit  in  the  KERA  reform  is  the  notion  of  progress,  that  each 
year's  students  ought  to  be  progressing  more  closely  toward  the  ideal  level  of  achievement.  Thus,  the  goals 
are  engineered  to  become  progressively  higher  with  each  school  year. 
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The  CDP  project  has  been  implemented  independently  of,  and  at  times  in  spite  of  KERA.  Where  KERA  is 
specific  on  goals  but  leaves  the  program  to  reach  those  goals  up  to  the  educators,  CDP  provides  a  concrete 
set  of  steps  to  make  education  better.  At  the  time  of  the  site  visit,  both  project  schools  had  been  cited  as 
"meeting  goals,"  while  one  of  the  two  comparison  schools  had  not.  While  not  billed  as  a  panacea  for 
KERA,  CDP  has  been  accepted  by  the  Jefferson  County  School  District  as  worthy  of  expansion  to  26 
elementary  schools  beyond  the  original  two  at  the  time  of  this  site  visit. 

(2)  Recess  time  for  children.  At  the  schools  visited  for  this  study,  in  part  because  of 
concerns  about  liability  for  children  left  unsupervised  on  the  playground,  the  current  school  policy 
encourages  teachers  to  take  their  children  outside  for  a  supervised  activity  for  20  minutes  during  the  day,  but 
otherwise  does  not  allow  time  periods  for  free  play.  While  this  policy  is  privately  opposed  by  many  CDP 
staff  from  California,  it  illustrates  the  limits  that  school-based  program  originating  from  outside  of  the  state 
must  abide.  Apparently,  the  same  policy  regarding  recess  is  in  effect  in  the  project  schools  in  Dade 
County,  Florida,  although  they  were  not  visited  in  doing  this  report.  "Because  the  children  have  grown  up 
not  knowing  about  recess,"  noted  one  staff  member,  "they  don't  know  what  they  are  missing."  There  are 
times,  however,  when  instruction  could  probably  be  more  effective  if  the  children  were  able  to  take  out  time 
for  free  play. 

(3)  Fidelity  to  the  project  model.  As  mentioned  above,  in  the  time  since  the  site  visit, 
Jefferson  County  District  has  chosen  to  expand  parts  of  the  model  to  20  additional  schools  and  all  of  the 
model  to  6  new  schools.  Because  the  behaviors  expected  on  the  part  of  teachers  are  complex,  and  a  degree 
of  encouragement  and  coaching  is  required,  the  rapid  expansion  will  test  the  abilities  of  the  I-Team 
members,  since  they  now  become  the  trainers,  and  the  administrative  staff  locally  and  in  Oakland.  While 
being  recognized  as  worthy  of  massive  expansion  is  certainly  commendable,  the  new  phase  will  test 
logistics  and  training  capacity  to  the  limits. 

CDP  Project  Video: 

This  introductory  video,  entitled  "Creating  Caring  Communities  of  Learners,"  differs  considerably  from  the 
two  previously  described.  Where  those  for  CLC  and  BBB  rely  upon  testimonials,  this  one  simply  shows 
children  learning  in  a  classroom  where  the  CDP  model  has  been  implemented.  The  close-up  views  of 
children  actively  engaged  in  learning  is  most  convincing,  with  camera  work  that  is  remarkable  in  catching 
of  subtle  elements  of  conversation  between  children  working  as  partners.  The  feeling  conveyed  is  of  a 
calm,  nurturing  atmosphere,  the  kind  of  setting  that  most  parents  would  want  for  their  child.  The  video 
leaves  the  observer  with  the  unstated  questions:  "Is  it  possible  for  every  child  in  school  to  have  this  kind  of 
positive  experience?"  and  "Could  educational  reform  really  provide  this  kind  of  education  for  all  children,  in 
a  setting  that  seems  to  minimize  the  negative  side  of  competition?"  While  a  major  portion  of  CDP  Project 
effort  has  gone  into  the  training  and  coaching  of  teachers,  the  video's  focus  upon  the  children's  learning 
shows  convincingly  what  the  ultimate  result  of  this  project  is  intended  to  be. 

Overall  communication  lessons. 

Several  lessons  can  be  drawn  from  a  close  analysis  of  the  introductory  videotapes  for  these  CSAP  projects: 

1 .  Prevalence  of  video.  As  a  beginning,  it  is  noteworthy  that  each  of  these  projects  had 
chosen  independently  to  develop  an  explanatory  video  about  their  project.  There  was  apparently  no  mandate 
from  Washington  or  program  incentive  of  any  kind,  but  the  leadership  of  each  of  these  projects  felt  the  need 
to  tell  its  own  story  using  a  video  format.  That  finding  may  say  something  about  the  expectations  of 
visitors  to  federally  funded  projects  in  the  1990's,  when  video  technology  is  within  the  range  of  most 
budgets  and  when  the  primary  information  source  for  many  people  is  television  and  video.  As  one  project 
administrator  stated:  "Our  videos  have  become  an  increasingly  important  part  of  our  efforts  to  describe  the 
'vision'  over  the  years.  We  have  never  been  able  to  convey  what  we  are  about  as  clerarly  in  words  as  it 
comes  through  seeing  it  'in  action.'"  Certainly  this  medium  allows  a  representation  of  the  project  in 
dimensions  that  are  not  possible  with  brochures  or  research  reports  (which  all  three  projects  had  as  well). 

2.  Impression  management.  The  project  videos  should  be  seen  as  a  form  of  "impression 
management"  that  all  of  us  engage  in,  putting  our  "best  foot  forward"  as  we  are  introduced  to  new  people  or 
shown  in  new  situations.  This  process  should  not  be  construed  as  being  dishonest,  but  rather  as  being 
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discrete  and  selective  of  the  details  to  present  in  introducing  the  project  and  in  what  order.  The  projects 
clearly  want  to  tell  their  stories,  and  they  do  so  by  assembling  a  collage  of  video  impressions  for  the 
newcomer.  They  are  accurate  in  their  portrayal  of  people  and  events  in  the  sense  that  they  attempt  to  be 
truthful,  but  producers  generally  work  to  convey  the  positive  aspects  of  their  projects.  The  negatives  and 
caveats  can  come  later,  for  the  person  who  cares  enough  to  investigate  in  more  detail. 

3.  Selection  of  issues  to  highlight.  The  issues  brought  out  in  the  videos  are  sometimes 
similar  but  often  very  different  from  those  reported  by  the  contract  ethnographer  after  receiving  a  week  or 
more  of  exposure  to  the  project  on  site.  In  only  one  case,  the  BBB  Project  in  Montana,  could  it  be  argued 
that  watching  the  project  video  had  some  influence  on  issue  selection  by  the  Site  Visitor,  since  the  viewing 
occurred  early  in  the  site  visitation  process.  Differing  views  on  the  issues  are  to  be  expected,  although  if 
the  aim  is  to  represent  the  project  from  an  "emic"  or  insider's  perspective,  one  would  expect  that  there 
would  be  some  congruence  between  issues  selected.  As  can  be  seen  from  Tables  1  and  2,  there  is  some 
convergence  in  issue  selection,  notably  with  the  BBB  Project,  but  possibly  more  divergence  in  issue 
selection  than  one  might  have  expected.  However,  in  some  cases,  the  differences  may  simply  reflect  word 
choice  and  semantics.  For  example,  for  the  CDP  Project  the  issue  of  schools  being  overly  competitive  is 
the  essential  question  behind  the  concern  with  the  Kentucky  School  Reform  Act  (KERA). 

4.  Emotional  tone.  In  each  case,  the  message  contained  on  the  video  provided  an  explanation 
but  had  clearly  been  designed  to  hook  the  viewer  emotionally:  in  CLC  with  the  power  of  honest  expression 
between  teenagers  and  their  parents,  in  BBB  with  the  gravity  of  alcohol's  devastating  effects  in  Native 
American  populations,  and  in  CDP  with  the  excitement  of  children  learning  cooperatively  in  an  enriched 
educational  environment.  There  is  little  that  could  be  called  "preaching".  The  people  speak  for  themselves 
in  convincing  ways.  It  would  be  difficult  to  watch  these  videos  passively,  without  experiencing  some 
emotion.  Each  in  its  own  way  works  at  persuading  the  viewer  that  the  perspective  of  this  project  to  AOD 
prevention  is  valuable. 


The  reports  described  herein  were  completed  with  funding  from  the  Center  for  Substance  Abuse  Prevention 
(CSAP).  The  authors  wish  to  express  appreciation  to  the  Project  staff  on  each  of  these  sites  for  their 
hospitality  and  openness,  as  well  as  permission  to  use  the  videos  from  their  respective  projects.  CSR  staff 
members  Anne  Kennedy,  Chris  Ringwold,  and  Ping  Yu  served  as  Site  Evaluators  to  the  various  sites,  and 
their  insights  shared  in  conversation  and  in  writing  were  invaluable  in  forming  these  conclusions.  And 
finally,  Drs.  Michael  Reed  and  Sherri  Aiken  were  instrumental  in  obtaining  the  permission  for  this 
reporting  to  occur. 

A  version  of  this  paper  is  being  submitted  for  journal  publication  in  Evaluation  and  Program  Planning. 

Additional  copies  of  the  article  are  available  by  request  from  Dr.  Nick  Eastmond,  Dept.  of  Instructional 
Technology,  Utah  State  University,  Logan,  UT  84322-2830.  Tel.  (801)  797-2642,  FAX  (801)  797-2693, 
or  Internet:  NEAST@CC.USU.EDU. 
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Figure   1:      Summary  of  Three   CSAP  Projects 


CLC  Project  BBS  Project  Child  Deuelopm't  Proj. 

Louisuiile,  Kentucky       Ronan,  Montana         Oakland,  California 


Sponsor- 
ing 
Agency 

Catholic  Diocese  of 
Greater  Louisuiile 
Area,  Protestant 
groups 

Confederated  Salfsh 
Kootenai  Tribal 
Council 

Oeueopmento!  Studies 
Center,  an  independent 
non-profit  agency 

Target 
Popul-a 
tion 

Youth  aged  12-14 
and  their  parents 

Members  of  9 
communities  on  the 
Flathead  Reseruation 

(adults  &  youth, 
Rnglo  &  Natiue  flmer) 

Students  in  grades  K-5 
and  their  teachers  in 

selected  schools 
(California,  Kentucky, 
Hew  York,  &  Florida). 

Inter- 
vention 

Working  through 
congregations, 
groups  of  teenagers 
&  their  parents  take 
part  in  workshops  on 
communicaton  skills, 
parenting,  BOD 
refusal  skills 

H  planning  group  for 

each  community 
designs  actiuities  for 
youth  &  parents: 
fostering  Natiue 
American  culture,  0- 

enhancing 
self-reliance  skills 

Teachers  recefue 
eHtensiue  training  in 
cooperetiue  learning, 
literature-based 

writing,  and 
developmental 
discipline.  They  teach 
using  these  skills. 

Issues 
raised 

—  The  need  for  a 
"safety  net"  after 
exposing  problems. 

—  The  effects  of  a 
strong  leader. 

—  Resiliency  of 
people  actiue  in  their 
church  group. 

—  The  fostering  of 
Natiue  American 
culture  in  small 
populations. 

—  Community/tribal 

diuisions. 

—  Effectiveness  of 
outsider  programs. 

—  interface  of  project 
with  KERR  (Kentucky 

Educational  Reform  Ret). 

—  Provision  of  recess 
time  for  children. 

—  Project  fidelity  to  a 
compleH  model,  esp. 

during  expansion  phase. 
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Figure    2:      Summary   of  findings   from   Project  Videos 


CLC  Project 
Louisville,  Kentucky 

BBS  Project 
Ronan,  Montana 

Child  Development 
Project:  Oakland,  Cfl 

Intended 
Rudience 

Neiu  participants  in 
Family  Networks 

U  of  Wisconsin  &  PBS 

iltamin  n  flllrtlonf^D 

LMoUJiny  OUQIBHl/C 

Stakeholders:  Board, 
fp«rhpr«  parents. 

Key 
elements 

Testimonials  from 

participants. 
Uiewers  participate 
in  ongoing  training 

program. 
Emotional  impact: 
Real  people  are 

Awareness  of 
dimension  of 
em'sting  problems: 
alcoholism,  suicide, 
denial,  shame, 
discrimination. 
Emotional  impact: 
pffpets  deuastatino. 

EHamples  of 
cooperatioe  learning 
in  younger  0/  older 
elementary  grades. 
Emotional  impact: 
soft,  soothing  pace 
can  lead  to  a  "caring 
community". 

Main 
Message 

ROD  preuention 
works,  esp.  when: 
—  planners  are 
professionals 

—  Genuine  caring 
and  sharing  exists 

—  Honest  effort 

Mtliiar  nfnhloimc 
%UlUcS  piUUicill** 

Solutions  come  from 
within  the  group  and 
are  based  upon 
Natlue  American 
traditional  ways. 
Tribal  history  Is  a 
source  of  pride  and 
Ann  rcsilfpncti. 

Cooperative  learning 
can  be  effectiue  if 
handled  by  a  skilled 
and  eating  teacher. 
These  strategies  help 
children  grow  up 
more  adjusted, 
healttui  &  haDDU. 

Issues 
raised 

—  Can  this  program 
be  effectiue  with  all 
kinds  of  people  (esp. 
shy  or  withdrawn)? 

—  How  long  will  the 
Project  be  prepared 
to  support  solutions 
to  fa  mi  in  oroblems? 

—  How  can  the  cycle 
of  negativity  & 

shame  he  broken? 

—  Can  outsiders  help 
a  situation  that  has 

been  the  result  of 
centuries  of  broken 
promises  &  neglect? 

—  is  present 
schooling  too 
competitive? 
--  How  con  children 
learn  from  uisual  & 
other  eHperlence? 
—  Can  school eHper- 
lence be  better? 

Special 
elements 

Group  meetings 
catch  the  emotion  of 
parents  and  children 
sharing  feelings 
about  each  other. 

The  emphasis  upon 
"healing"  provides  a 
more  posltiue  basis 
for  action.  The  need 
for  adults  to  "model 

appropriate 
behauior"  for  youth 
1    comes  out  clearly. 

Close-in  uiews  of 
children  working  on 

engaging  lessons 
provides  eHcitement 
with  their  curiosity 
and  basic  innocence. 
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ABSTRACT 


This  paper  summarizes  the  results  of  a  study  examining  the  implementation  of  a  computer-based  trainer  in  a 
US  Army  Advrnced  Individual  Training  center.  Interviews,  document  analysis,  and  participant-observation 
were  used  to  identify  factors  that  influenced  potential  adopters  to  accept  or  resist  the  new  trainer. 
Comments  from  planners  &  administrators,  technical  support  personnel,  subject-matter  experts 
(instructors),  instructional  developers,  and  students  were  analyzed  for  common  themes  affecting  the 
adoption/rejection  decision.  Nineteen  factors  were  identified,  focusing  on  different  situational  entities  (i.e., 
other  participants,  the  system  itself,  external  organizations,  leaders  of  the  implementation,  and  the 
bureaucracy).  Results  suggest  four  critical  issues  for  planners  to  consider  to  maximize  the  positive  impacts 
of  each  factor  throughout  trie  dissemination  process.  Recommendations  for  addressing  these  issues  are 
offered  to  administrators,  instructional  technologists,  and  change  agents. 


INTRODUCTION 

United  States  Army  training  centers  have  used  instructional  technology  throughout  their  curricula 
for  many  years.  This  was  a  natural  development  in  an  organization  dealing  with  many  technical  disciplines 
rooted  in  electronics.  Frequently,  dissemination  of  this  technology  proceeded  smoothly,  and  promoted  more 
effective  learning.  Sometimes,  however,  it  produced  failures  as  spectacular  as  any  in  the  private  sector. 
Likewise,  as  in  business  and  academe,  initial  attributions  of  these  failures  to  poor  system  quality  or 
incompetent  personnel  ultimately  fell  under  the  onslaught  of  contrary  data.  Since  previous  research 
focusing  on  instructional  effectiveness  and  other  system  characteristics  was  often  inconclusive  (sometimes 
giving  high  marks  to  a  system  that  failed  miserably),  another  approach  was  needed. 

The  approach  selected,  diffusion  research,  recognizes  that  even  innovations  of  proven  effectiveness 
can  fail  if  their  intended  users  will  not  commit  to  their  success  (Wellin,  1955;  Mosteller,  1981;  Rogers, 
1983).  Thus,  it  is  concerned  with  the  human  factors  which  facilitate  or  hinder  this  commitment. 

The  diffusion  paradigm  breaks  down  some  of  the  philosophical  barriers  between  qualitative  and 
quantitative  methods.  Its  roots  are  qualitative,  and  it  often  makes  use  of  in-depth  interviews,  document 
analysis,  or  participant-observation  (Rogers,  1983).  These  tools  are  especially  useful  in  this  context,  since 
the  objective  is  to  gain  insight  into  the  concerns  of  the  individuals  who  interact  with  the  innovation.  At 
the  same  time,  diffusion  research  does  not  dismiss  quantitative  measures,  often  using  correlation  to 
associate  independent  variables  with  innovativeness,  rate  of  adoption,  etc.  Likewise,  diffusion  scholars  may 
use  statistical  techniques  to  analyze  possible  trends  represented  by  their  data  (Fullan  and  Pomfret,  1977; 
Berman,  1981). 

Many  studies  of  both  types  have  been  conducted,  providing  a  rich  framework  for  comparison  that 
can  help  overcome  the  weak  generalizability  associated  with  case  st  do  the  key  factors  in  a  military  training 
setting  differ  from  those  of  the  civilian  educational  settings  explored  in  past  research?  This  required 
comparison  of  the  findings  to  existing  frameworks. 

METHOD 

This  study  employed  a  qualitative  perspective.  The  adoption/rejection  decision  is  one  that  is 
intensely  personal  (Rogers,  pp.20,  21 ).  Even  if  it  is  assumed  that  the  same  set  of  factors  underlies  it  for  all 
people,  the  way  each  individual  perceives  those  factors  is  likely  to  vary  considerably.  Since  it  is  ultimately 
the  potential  adopter,  not  the  change  agent,  who  will  decide  whether  to  adopt  or  reject  an  innovation,  this 
study  had  to  examine  these  individual  perceptions  to  offer  meaningful  guidance  to  future  dissemination 
efforts  (Burkman,  pp.439,  440,  442). 

Unfortunately,  a  quantitative  approach  to  this  examination  was  problematic.  Statistical  measures 
of  observed  behaviors  or  results  are  almost  certain  to  be  more  representative  of  the  researcher's  cognitive 
structure  than  the  subjects'  because  it  is  the  researcher  who  selected  the  variables  to  be  measured.  Similarly, 
surveys  may  pose  questions  in  a  way  that  does  not  correspond  to  the  subjects'  way  of  thinking  about  such 
things,  and  may  have  difficulty  detecting  behaviors  that  the  subjects  would  not  be  inclined  to  brag  about. 
Perhaps  most  important,  for  the  study  to  identify  interrelationship?  between  factors,  it  was  essential  that 
factors  be  described  in  the  potential  adopters'  own  terms,  as  the  relationships  of  interest  were  those  that 
existed  in  their  minds.  A  qualitative  approach  offered  the  most  effective  means  of  accomplishing  this  end 
(Martin,  1988,  pp.3,  4). 


Participant  Selection 

One  might  imagine  that  what  is  important  might  vary,  depending  on  each  individuals'  role  relative 
to  the  trainer.  To  test  this  hypothesis  (and  to  protect  reliability  from  its  potential  effects),  the  participants 
were  selected  from  the  categories  suggested  by  Garland  (1991 ,  p.255).  These  include  management, 
information  systems  technologists,  subject  experts,  instructional  designers,  and  learners.  For  each  category, 
individuals  were  selected  whose  relationship  to  the  BMMT  fell  within  that  description.  These  people  were 
approached  about  being  interviewed  and,  as  part  of  this  conversation,  asked  who  else  was  involved  with  the 
device  when  it  first  got  started.  This  procedure  was  repeated  until  no  new  names  were  generated.  This 
completed  the  participant  selection  process,  except  for  students,  and  generated  twenty-two  names. 

Representing  the  student  perspective  proved  more  troublesome.  The  procedure  employed  to  select 
other  participants  was  intended  to  obtain  100  percent  coverage  of  those  personnel  still  present.  For 
students,  this  was  neither  possible  (the  last  ones  having  graduated  well  before  the  study  began)  nor  practical 
(literally  hundreds  being  present  at  any  one  time);  consequently,  an  alternative  selection  strategy  was  used. 

Selection  of  students  began  with  the  assumption  that  their  perspective  remains  relatively  constant 
across  time  (the  system  is  always  new  to  the  students).  The  major  weakness  of  this  assumption  is  that, 
during  implementation,  the  students  were  faced  with  instructors  for  whom  the  system  was  also  new.  This 
threat  was  countered  by  including  participant-observation  logs  from  four  trained  observers  (including  the 
author)  who  participated  in  early  classes.  Actual  students  were  then  selected  using  a  stratified  random 
sampling  technique,  with  two  selected  from  each  available  class,  for  a  total  of  sixteen.  One  selected  student 
from  each  class  had  used  the  resource  suite  (a  source  of  remedial  instruction),  while  the  other  had  not. 
Together,  these  methods  were  intended  to  ensure  that  perspectives  were  provided  that  reflect  the  full  range  of 
instructor  teams  and  instructional  approaches  for  the  system.  They  also  ensured  that  students  who  had 
trouble  with  the  course  were  represented  equally  with  those  whose  achievement  met  or  exceeded  normal 
levels. 

Data  Collection 

As  already  implied,  this  study  is  an  historical  retrospective,  as  the  BMMT  was  implemented 
approximately  twc  years  before  it  began.  Since  this  prevented  the  use  of  participant-observation  (other  than 
that  recorded  in  historical  documents),  the  primary  method  of  data  collection  was  the  in-depth  interview. 
This  method,  as  described  by  Bogdan  &  Biklen  (1992.  pp.2,  3,  96-101)  uses  a  few  general  questions  to 
orient  the  discussion,  but  generally  allows  the  interviewee  to  talk  about  what  (s)he  sees  as  important.  That 
is,  after  all,  the  objective  (Martin,  p. 6). 

The  most  specific  guidance  was  reserved  for  the  introduction,  and  for  the  demographic  questions. 
Before  starting  this  portion  of  the  interview,  a  few  minutes  of  shop  talk  were  used  to  help  relax  the 
participant,  and  to  make  the  interview  less  like  formal  questioning  and  more  like  an  everyday  conversation. 
The  interview  itself  consisted  of  two  core  questions:  Tell  me  about  your  involvement  with  the  BMMT 
when  it  first  got  started,  and  Tell  me  about  yourself.  Once  each  question  was  asked,  the  interviewee  was 
allowed  to  talk  everything  out  before  a  series  of  more  specific,  semi-structured  questions  under  each  core 
question  were  asked.  The  use  of  these  questions  helped  in  the  comparison  of  responses  from  different 
participants.  The  interview  ended  when  the  interviewee  ran  out  of  things  to  say.  Demographic  questions 
were  reserved  for  last  so  the  interviewee  would  be  more  relaxed.  This  strategy  was  selected  under  the 
assumption  that  beginning  with  demographic  items  would  focus  the  interviewee  on  the  fact  that  this  was  a 
research  interview,  perhaps  altering  the  nature  of  the  conversation. 

To  help  ensure  accurate  representation  of  each  interview,  a  tape  recording  was  made  of  the  session 
(after  securing  the  interviewee's  permission).  For  those  who  choose  not  to  be  recorded,  field  notes  were 
taken  directly.  In  either  event,  the  interviewee  was  assured  that  all  data  collected  would  be  held  in 
confidence,  and  would  not  be  reported  with  names  or  other  identifying  information. 

The  major  weakness  of  an  interview  strategy  under  these  circumstances  is  that  it  relies  on 
participants'  recollection  of  events  and  their  reactions  to  them.  This  required  the  use  of  additional  data 
sources  for  triangulation.  Fortunately,  the  military  bureaucracy  is  well  known  for  generating  a  volume  of 
historical  documents,  which  served  as  a  rich  data  source  representing  several  points  of  view.  The  most 
common  of  these  was  the  official  point  of  view  of  the  school  hierarchy.  Documents  originated  at  each 
echelon  provided  a  glimpse  at  coordination  that  occurred  during  the  process,  and  various  viewpoints  on  the 
device  itself.  Besides  the  official  perspective,  records  of  the  evaluation  organization  within  the  school 
provided  professional-grade  qualitative  data  from  focus  groups  conducted  during  implementation.  As  these 
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records  described  the  feelings  and  concerns  of  participants  during  the  implementation,  recorded  at  that  time, 
they  served  as  a  useful  cross-check  for  interview  data  collected  during  this  study. 

Coding  and  Analysis 

Some  degree  of  ireflective  analysis  during  data  collection  was  inevitable,  as  analysis-in-the-field  is 
an  essential  part  of  the  qualitative  tradition  (Bogdan  &  Biklen,  p.  154).  This  consisted  chiefly  of  the 
emergence  of  coding  categories  during  the  study,  leading  to  increased  attention  paid  to  particular  comments 
occurring  during  an  interview  comments  that  frequently  reinforced  a  tentative  category  or  suggested  the 
emergence  of  a  new  one.  Occasionally,  such  informal  analysis  also  alerted  the  author  to  additional 
questions,  new  data  sources,  or  differing  perspectives  of  potential  adopters. 

Primary  data  analysis,  however,  occurred  toward  the  end  of  the  study.  In  part,  this  was  to  preserve 
energy  and  time  for  the  tasks  of  establishing  rapport  and  getting  on  in  the  field,  as  suggested  by  Bogdan  & 
Biklen  (ibid.);  in  part,  it  was  to  ensure  that  the  perspectives  gained  from  analyzing  early  interviews  did  not 
limit  the  scope  of  inquiry  in  later  ones. 

This  analysis  process  began  with  transcription  of  interviews.  At  this  time,  the  tape  recorded  (or 
manually  recorded,  via  field  notes)  interviews  were  entered,  with  the  various  classes  of  documents  discussed 
earlier,  into  journal  format  on  a  computer.  The  data  were  then  reviewed,  as  a  whole,  to  formalize  the  initial 
coding  categories  discovered  during  data  collection.  This  use  of  open  coding  facilitated  the  broadest  coverage 
of  the  data. 

Once  the  core  categories  were  extracted  in  this  fashion,  the  study  shifted  to  axial  coding  to  explore 
them  in  greater  depth.  This  portion  of  the  process  focused  on  identification  of  examples  for  each  category 
from  the  data,  and  occasionally  the  perception  of  new  core  categories  that  more  closely  fit  the  observed 
phenomena. 

RESULTS 

Results  of  the  study  fall  into  two  sections,  associated  with  initial  and  subsequent  rounds  of 
analysis.  In  the  first  round,  factors  extracted  from  participant  comments  were  organized  into  categories,  and 
general  impressions  of  findings  were  formed.  In  the  final  round,  these  categories  and  impressions  were 
examined  to  identify  major  themes,  or  critical  issues,  that  could  be  acted  upon  by  planners  and 
administrators. 

Factors  and  Factor  Groups 

Initial  coding  identified  nineteen  factors  comprising  five  major  groups.  These  focused  on  how 
participants  perceived  certain  characteristics  of  the  different  entities  involved  in  the  implementation.  Such 
entities  included  other  participants,  the  system  itself,  external  organizations,  leaders  of  the  implementation, 
and  the  bureaucracy. 

Participant  Factors 

These  factors  describe  how  potential  adopters  own  characteristics,  and  beliefs  about  those  of  others, 
affected  their  perceptions  of  the  system.  Participant  type  refers  to  the  role  of  the  individual  with  respect  to 
the  innovation  (e.g.,  management,  instructors,  etc.).  Some  comments  reflected  participants  views  of  the 
effect  of  others  roles  on  their  perceptions  (e.g.,  The  command  group  just  didn't  understand  what  the 
instructors  needed  this  thing  to  do).  As  expected,  however,  its  greatest  impact  was  the  effect  of  each 
participants  own  role  on  the  factors  that  affected  them.  The  individual's  role  was  also  found  to  impact  less 
on  what  factors  were  important  than  on  how  those  factors  were  viewed.  For  example,  individuals  from  all 
roles  talked  about  system  quality  and  gains  (or  losses)  associated  with  adoption.  In  doing  so,  however, 
senior  personnel  focused  on  its  impact  on  students,  while  mid-level  personnel  emphasized  their 
responsibilities  for  the  innovation  itself.  Instructors  highlighted  its  effects  on  their  interaction  with  other 
individuals,  especially  students,  while  students  themselves  considered  what  the  innovation  required  of  them. 
Other  factors  in  this  group  included  participant  background,  referring  to  whether  an  individual  was 
considered  a  Morse  subject-matter  expert,  competence,  describing  perceptions  of  co-workers  abilities  to 
perform  their  system-related  duties,  and  views  of  change,  relating  to  a  participants  orientations  and  reactions 
to  change  overall. 
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System  Factors 

This  second  set  of  factors  addressed  potential  adopters'  views  of  the  trainer  itself.  Quality  refers  to 
a  participant's  perception  of  the  trainers  merit  (e.g.,  poor,  fair,  good).  Net  Gain  describes  perceptions  of 
this  merit  in  relation  to  that  of  the  previous  system  (e.g.,  worse,  equivalent,  better).  These  factors  were 
surprisingly  distinct,  i.e.,  several  participants  felt  that  BMMT  was  a  poor-to-fair  system  because  it  coddled 
students,  yet  these  same  individuals  described  it  as  much  better  than  its  predecessor,  and  therefore  supported 
it.  A  final  factor  of  this  type,  lessons  learned,  refers  to  a  participantfs  impression  cf  how  the  teachings  of 
experience  were  incorporated  into  the  new  system  and  its  surrounding  philosophy. 

External  Organization  Factors 

Another  common  theme  described  dealings  with  outside  organizations,  such  as  hardware  and 
software  vendors.  Capability  describes  a  participants  views  of  the  outside  organizations  technical  capacity 
to  perform  according  to  the  stated  requirements.  Motivation  covers  perceptions  of  the  goals  that  guided  the 
outside  organizations  planning  and  actions  (e.g.,  the  most  profit  with  the  least  effort  vs.  genuine  concern 
for  improving  the  learning  process).  Finally,  attitude  refers  to  a  participants  impression  of  the  tone  of 
interaction  with  representatives  of  the  outside  organization  (e.g.,  condescending,  hostile,  cooperative). 

Leader  Factors 

One  of  the  most  pervasive  factor  groups  relates  to  the  characteristics  of  the  implementation  efforts 
leadership,  as  seen  by  the  potential  adopters.  Continuity  describes  participant  views  on  the  consistency  of 
the  implementation  strategy  as  the  effort  progressed.  Comments  showed  that  personnel  at  all  levels  had 
difficulty  adjusting  to  the  implementations  new  personality  when  key  leaders  retired  or  were  reassigned 
during  the  effort.  Qualifications  refers  to  a  participants  awareness  of  the  background  of  the  implementations 
leadership,  and  how  they  felt  this  affected  leaders  ability  to  manage  the  process.  Involvement  addresses  the 
potential  adopters  perceptions  of  the  leaderships  level  of  interest  in  and  attention  to  the  implementation. 

Bureaucracy  Factors 

This  final  set  of  factors  reflects  the  Defense  bureaucracies  artificial  compartmentalization  of  the 
change  process,  as  experienced  by  potential  adopters.  Most  individuals  reported  feeling  isolated  from  the 
other  organizations  involved  throughout  the  process.  In  fact,  this  contributed  to  the  focus  of  the  factor 
groups,  as  individuals  struggled  to  make  sense  of  the  implementation  effort  as  a  whole,  while  seeing 
themselves  as  cut  off  from  all  but  the  adjacent  parts.  These  categories  tended  to  correspond  to  the  phases  of 
the  Systems  Approach  to  Training  (SAT)  model.  Analysis  describes  participant  views  on  die  effectiveness 
of  the  front-end  analysis.  Comments  suggested  that  instructors  and  training  division  administrators 
interpreted  this  as  job  and  task  analysis,  and  were  concerned  that  no  one  ever  took  the  time  to  look  at  what 
we  do  down  here  and  figure  out  how  BMMT  could  support  it.  Senior-level  command  and  staff,  on  the  other 
hand,  talked  about  needs  assessment  as  described  by  Kaufman  (1972):  how  does  the  status  quo  differ  from 
the  desired  end-state,  and  what  must  the  BMMT  do  to  move  us  from  here  to  there  and  felt  that  the  trainers 
obvious  efficiency  at  doing  so  meant  the  process  had  been  successful.  Design  refers  to  participant 
understanding  of  the  process  used  to  generate  the  requirements  for  the  system.  Development  relates  to  their 
perceptions  concerning  creation  of  the  courseware.  Implementation  describes  their  feelings  about  the 
adequacy  of  guidance  they  received  or  ideas  they  were  allowed  to  contribute  as  training  with  the  system 
began.  Finally,  departing  from  the  SAT  phase  factors,  instruction  of  key  personnel  relates  to  participant 
assessments  of  the  process  used  to  familiarize  them  with  its  operation,  and  coordination  describes  their 
perceptions  of  how  it  all  was  (or  was  not)  tied  together. 

Critical  Issues  and  Recommendations 

A  final  round  of  analysis  focused  on  translating  the  factors  identified  by  participants  into  critical 
issues  that  administrators,  instructional  technologists,  and  change  agents  could  have  focused  on  during  the 
planning  stages  of  the  trainer  to  smooth  its  implementation.  Many  factors  were  eliminated  at  this  time, 
reflecting  differences  between  the  participant  perspective  and  that  of  the  planner.  This  is  not  a  judgment  of 
validity;  it  is  the  product  of  an  audience  analysis.  One  senior  administrator  put  it  this  way:  Don't  waste 
my  time  telling  me  what  I  already  know;  don't  waste  my  time  telling  me  what  I  can't  change;  don't  waste 
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my  time  telling  me  what  doesn't  matter  This  guidance  defines  a  critical  issues  by  exclusion.  Some  factors 
eliminated  may  be  considered  common  sense  (e.g.,  the  effectiveness  of  the  new  system,  by  itself  or  relative 
to  the  old  system);  others  are  outside  the  organizations  control  (e.g.,  the  capability,  motivations,  or  attitude 
of  the  contractors).  Many  more,  while  they  appeared  in  several  sources,  were  neither  pervasive  throughout 
the  data  nor  particularly  critical  to  those  who  mentioned  them.  The  four  issues  discussed  here  are  those  that 
represent  central  themes  throughout  the  data,  could  have  been  implemented  by  planners  and  administrators 
in  the  organization,  and  would  probably  have  promoted  acceptance  at  the  least  cost  to  resources  or  effort 
(based  on  participant  comments). 

Coordination 

This  issue  is  grounded  in  the  leader  factor  involvement  and  the  bureaucracy  factor  coordination. 
Taken  together,  these  factors  describe  a  participant's  perception  of  the  role  played  by  the  organization's 
leadership  in  planning  and  managing  the  change  process,  and  in  keeping  their  subordinates  informed  as  that 
process  progressed.  Participants  especially  those  at  lower  echelons  saw  those  in  key  leadership  positions  as 
uninvolved  and  uncommunicative.  Planners  and  administrators  were  generally  described  as  lacking 
knowledge  of  project  specifics,  or  of  technical  background.  Participants  seldom  heard  anything  about  what 
was  happening  outside  their  level  of  their  organization.  This  contributed  to  a  common  perception  that  no 
one  was  at  the  wheel  and  that  other  organizations  either  were  not  working  much  at  all  or  were  working  at 
cross  purposes.  For  their  part,  planners  and  administrators  interviewed  were  clearly  taking  active  roles  in 
the  project,  and  took  a  certain  amount  of  pride  in  their  grasp  of  its  complexities.  In  fact,  their  comments 
often  reflected  a  belief  that  the  concerns  of  those  at  lower  echelons  stemmed  from  lack  of  exposure  to  the 
big  picture 

This  is  clearly  a  communications  issue.  The  structure  of  this  type  of  organization  discourages 
information  flow  between  other-than-adjacent  echelons.  Unfortunately,  while  it  is  a  necessary  part  of 
military  order  and  discipline,  its  strict  interpretation  does  promote  the  sort  of  perception  discussed  above  and 
restricts  the  ability  of  the  organization  to  bring  its  collective  experience  to  bear.  To  optimize  both  these 
criteria,  the  organization's  leadership  might  sponsor  some  form  of  structured  information  flow  outside 
conventional  channels.  One  participant  suggested  a  project  newsletter,  published  by  the  project's  lead 
organization  and  actively  soliciting  contributions  from  all  levels  of  all  organizations  involved.  This  low 
cost  alternative  would  provide  a  quick  and  convenient  forum  for  participants  to  share  ideas  and  keep  each 
other  current  on  the  activities  of  their  particular  section.  It  would  offer  the  dual  benefit  of  increased 
goodwill  and  pooling  of  expertise  on  all  phases  of  development. 

Empowerment 

This  second  issue  is  grounded  in  the  bureaucracy  factors  analysis,  design,  and  implementation. 
The  intended  users  of  a  new  trainer  are  primarily  personnel  from  the  academic  department  for  which  it  is 
intended.  Unfortunately,  the  findings  showed  that  actual  (future)  users  of  the  trainer  were  almost  never 
consulted  or  involved  in  requirements  definition.  Comments  from  those  who  drew  up  the  specifications  for 
them  often  showed  this  to  result  from  their  perceptions  that  they  knew  what  the  users'  requirements  would 
be  (often  because  they  had  once  held  the  users'  roles).  Whether  this  assumption  was  valid  or  not,  it  fostered 
the  impression  that  those  in  charge  did  not  place  much  value  on  what  the  trainer's  future  users  thought  or 
wanted.  Thus,  in  a  worst  case  scenario,  a  system  could  be  designed  based  on  inaccurate  perceptions  of  user 
needs  and  the  users  could  know  this  and  resent  the  system  (and  its  proponents)  for  it.  This  in  turn  would 
encourage  users  to  magnify  the  system's  failures,  and  to  blame  them  on  their  lack  of  involvement  in 
defining  the  requirements  in  the  first  place. 

To  prevent  this,  planners  and  administrators  should  ensure  the  active  representation  of  actual 
(future)  users  throughout  the  implementation  process.  This  is  most  critical  for  instructors,  who  will  have 
the  most  day-to-day  contact  with  the  system  over  an  extended  period  (and  thus  the  most  impact  on  its 
success  or  failure).  Again,  this  does  not  need  to  be  incompatible  with  a  structured  environment.  A  training 
division  administrator  can  be  asked  to  task  one  of  his  instructors  as  the  project  team  representative.  The 
finding?  ^owed,  however,  that  this  person  must  continue  to  be  primarily  an  instructor.  Otherwise,  his 
understanding  of  instructor  priorities  may  become  dated,  and  he  will  cease  to  be  seen  as  a  peer,  becoming 
instead  a  project  team  stoolie.  Similarly,  such  an  individual  must  confer  frequently  with  his  fellows,  to 
ensure  that  he  is,  in  fact,  representing  them  and  not  like  the  former  user  in  the  preceding  paragraph  merely 
assuming  he  understands  them.  Administrative  and  maintenance  personnel  who  will  personally  use  the 
system  may  be  represented  in  the  same  manner.  Student  representation  may  generally  be  limited  to 
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observations  or  interviews  during  the  front-end  analysis,  and  conventional  testing  for  usability  and 
instructional  effectiveness. 

Competence 

This  issue  is  grounded  in  the  participant  factor  competence,  the  external  organizadon  factor 
capability,  and  the  leader  factor  qualifications.  The  importance  of  this  issue  is  probably  best  known  in 
relation  to  subject-matter  (or  other  contextual)  expertise.  This  is  due  to  the  prevalence  of  external 
contractors  as  development  agencies  for  the  type  of  system  discussed  here.  While  contractor  personnel  are 
usually  competent  developers,  they  generally  lack  the  same  familiarity  with  the  subject  matter  being  taught 
and  the  context  in  which  it  will  be  applied  that  is  associated  with  internal  personnel.  However,  it  can  be 
equally  crippling  when  the  perceived  deficiency  in  competence  relates  to  the  development  process  itself. 
Likewise,  if  the  project  team  is  seen  as  lacking  the  authority  (a  sort  of  capability)  to  enact  its  decisions,  no 
amount  of  subject-matter  or  development  expertise  will  compensate  for  this.  While  each  is  necessary, 
neither  subject-matter  experts,  nor  instructional  developers,  nor  skilled  leaders  are  sufficient  without  the 
others  to  produce  an  effective  system. 

Consequently,  planners  and  administrators  should  ensure  that  all  required  skills  are  represented 
when  a  project  team  is  formed.  That  participants  perceive  this  to  be  the  case  is  at  least  as  important.  The 
findings  offer  an  example  where  the  military  courseware  developers  possess  roughly  the  same  mix  of 
development  skills  and  experience  as  would  be  allocated  to  a  project  of  comparable  size  by  a  contractor. 
However,  since  they  did  not  initially  possess  this  experience,  they  were  written  off  as  unqualified  by  many 
other  participants.  Thus,  planners  and  administrators  should  attempt  to  select  project  team  members  whose 
expertise  in  their  area  is  respected  by  other  participants  especially  the  system's  intended  users. 

Personal  Benefit 

This  final  issue  is  grounded  in  the  system  factors  equality  and  net  gain  and,  to  a  lesser  extent,  the 
bureaucracy  factor  instruction  of  key  personnel  It  has  two  major  components.  The  first  asks,  How  will  use 
of  the  new  system  affect  how  well  or  how  easily  I  can  do  my  job  The  second  asks  How  will  use  of  the  new 
system  affect  the  nature  or  status  of  my  job  Comments  from  participants  at  all  levels  suggested  that  the 
BMMT"s  implementation  effort  was  aided  by  both  aspects.  The  trainer  was  seen  as  offering  substantial 
improvements  in  efficiency  and  effectiveness  of  job  performance,  and  most  participants  reported  some  level 
of  personal  benefit  associated  with  these  gains. 

Planners  and  administrators  should  understand  that  this  may  not  always  be  the  case.  The  BMMT 
replaced  a  similar,  computer-based  system  that  nearly  all  participants  felt  needed  replacing.  User  roles  were 
not  substantively  redefined,  nor  did  users  feel  personally  threatened  by  the  new  system.  Systems  introduced 
into  classrooms  previously  dominated  by  lecture/conference  instruction  and  practical  exercises  may  leave 
users  unsure  of  how  they  should  interact  with  the  new  trainer,  and  threatened  by  its  intrusion  into  a  familiar 
environment  with  comfortable  patterns  of  control.  This  can  be  aggravated  by  a  change  agent  who  stresses 
the  strengths  of  a  new  trainer  relative  to  conventional  instruction,  and  is  thus  perceived  as  presenting  it  as  a 
more  efficient  alternative. 

This  can  create  a  false  dichotomy  between  traditional  and  computer-based  instruction.  In  reality, 
neither  is  a  philosophy  to  be  accepted  or  rejected.  Both  are  tools  for  an  instructor  which,  like  any  other 
tools,  are  appropriate  for  some  tasks  and  inappropriate  for  others  and  both  may  be  used  together.  For 
example,  complex,  conceptual  topics  may  require  the  adaptive  skills  of  a  human  instructor  and  be  best 
suited  to  a  lecture/conference  mode  while  basic  declarative  knowledge,  or  psychomotor  or  procedural  skills 
may  require  extensive,  self-paced  drill-and-practice  and  be  best  suited  to  computer-assisted  delivery. 
Furthermore,  the  conceptual  topic  mentioned  above  may  have  to  be  integrated  into  a  sequential  procedure, 
suggesting  a  lecture/conference  followed  by  a  computer-assisted  practical  exercise.  Decision-makers  should 
emphasize  this  toolbox  orientation,  whether  they  are  integrating  a  training  device  into  an  existing  course  or 
designing  a  new  course  with  device-based  support.  A  strategy  that  recognizes  the  instructor  as  the  central 
component  of  all  resident  instruction  is  likely  to  meet  with  much  more  enthusiasm  than  one  that  proposes 
a  device  or  system  as  the  new  center  of  attention.  Technological  support  should  instead  be  introduced  as  a 
means  of  relieving  the  instructor  of  tedious,  repetitive  administrative  chores  that  a  computer  can  perform 
more  efficiently  thereby  freeing  him  to  address  the  real  business  of  teaching.  An  important  part  of  this 
approach  must  also  be  provision  of  transition  training,  to  provide  instructors  with  tips  and  techniques  for 
using  the  system  to  their  greatest  advantage,  as  these  techniques  are  frequently  quite  different  from  those 
used  in  unassisted  platform  instruction. 
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DISCUSSION 


Beginning  with  its  central  question,  this  study's  findings  can  be  summarized  as  follows: 
triangulation  from  the  three  data  sources  (interviews,  historical  documents,  and  participant-observation) 
produced  a  set  of  robust  factors  offering  a  credible  explanation  for  participants  adoption/rejection  decisions. 
As  expected,  the  BMMT  as  a  successful  system  provided  a  wealth  of  positive  comments  that  reflected  a 
solid  implementation  strategy.  More  surprisingly,  it  also  offered  many  lessons  learned,  pointing  out  factors 
that  were  not  addressed  to  the  satisfaction  of  most  potential  adopters. 

Taken  together,  these  positive  and  negative  impressions  represent  the  importance  of  each  factor  to 
participants  ...  and  thus  the  value  of  attention  paid  to  that  factor  as  a  force  multiplier  for  increasing  the 
systems  chances  of  acceptance.  Considered  separately,  the  vector  sum  of  these  impressions  represents  the 
degree  to  which  planners  and  administrators  already  understand  and  attend  to  each  factor.  (A  strong,  positive 
vector  sum  shows  the  factor  is  under  control;  a  strong  negative  suggests  it  is  not,  with  a  range  of 
possibilities  between).  By  applying  these  criteria,  together  with  an  assessment  of  the  organization's 
leverage  on  each  factor,  this  study  was  able  to  derive  a  set  of  critical  issues  which  deserve  additional 
attention.  (This  approach  may  be  useful  even  in  situations  where  these  particular  factors  are  less  prevalent.) 

Moving  on  to  the  first  related  question,  the  initial  hypothesis  concerning  the  effect  of  a 
participant's  role  relative  to  the  trainer  was  confirmed  by  the  findings.  Its  effect,  however,  was  primarily 
one  of  perspective  on  each  factor,  rather  than  determining  the  factors  themselves.  Furthermore,  based  on 
analysis  of  the  types  of  comments  made  by  each  participant,  some  migration  occurred  from  Garland's 
categories.  As  a  result,  several  participants  finished  the  study  in  a  category  other  than  that  for  which  they 
were  selected.  Redesignation  of  the  categories  according  to  their  new  membership  resulted  in  the  following 
perspectives:  management,  administrative  staff,  instructors,  and  students. 

Concluding  with  the  final  question,  comparison  with  previous  studies  showed  widespread 
congruence.  Differences  that  arose  reflected  viewpoint  more  than  substance,  and  illustrated  key  effects  of  the 
military  setting.  For  example,  the  traditional  category  of  training  effectiveness  was  frequently  viewed  as  a 
negative  factor  because  students  who  are  also  soldiers  are  expected  to  succeed  in  spite  of  the  worst  the 
environment  can  throw  at  them.  Table  1  summarizes  the  correspondence  between  factors  identified  in  this 
study  and  three  prominent  frameworks  based  on  research  in  other  settings.  (Each  X  represents  one 
corresponding  factor.) 
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TABLE  1   Support  for  Factors  from  Research  in  Other  Settings 


Identified  Factor 

Rogers  (1983) 

Ely  (1990) 

Zaltman  &  Duncan  (1977) 

Participant  Type 

XXX 

Participant  Background 

X 

X 

XX 

Competence 

X 

X 

Views  of  Change 

X 

XXX 

XXXXX 

Quality 

XXX 

Net  Gain 

XX 

XXX 

XXXXXX 

LessonsLeamed 

X 

Capability 

X 

X 

Motivation 

XXXX 

Attitude 

XXX 

Continuity 

XX 

Qualifications 

X 

XX 

Involvement 

XXXXX 

XX 

Analysis 

X 

X 

Design 

X 

X 

Development 

X 

X 

Implementation 

X 

X 

Instruction  of  Key  Personnel 

X 

XX 

XX 

Coordination 

XX 

The  scientific  impacts  of  this  study  follow  from  this  comparison.  Despite  the  environmental 
differences  associated  with  a  military  setting,  the  factors  that  the  study  identified  corroborated  those  of  its 
civilian/educational  predecessors.  This  supports  the  possibility  of  a  diffusion  paradigm  that  is,  at  some 
level,  applicable  across  all  settings.  At  the  same  time,  the  different  viewpoint  of  the  results  adds  richness 
to  the  research  base  through  examples  of  setting-induced  variance. 


Limitations  of  the  Study 

The  most  critical  limitation  of  this  study  results  from  its  status  as  a  single-site  case  study.  Of  all 
qualitative  techniques,  this  type  ot  study  offers  the  least  generalizability  due  to  its  focus  on  a  particular 
situation.  Other  threats  to  external  validity  (generalizing  power)  include  its  use  of  a  military  site,  with  the 
associated  centralized,  highly  cohesive  structure.  While  the  support  drawn  from  previous  research  in  other 
settings  offers  some  assurance  that  similar  factors  may  be  encountered  elsewhere,  the  findings  reported  will 
be  of  primary  use  only  to  suggest  issues  for  consideration,  or  frameworks  for  future  inquiry. 

It  should  also  be  remembered  that  this  study's  objectives  were  restricted  by  the  research  questions. 
Factors  were  identified  and  used  to  generate  issues  for  consideration,  and  these  issues  were  examined  for  the 
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effect  of  participants'  roles  relative  to  the  system  (and  compared  with  frameworks  from  research  in  other 
settings).  Beyond  this,  e  ^ioration  of  interrelationships  between  factors,  or  statistical  investigation  of 
correlations  between  facto*.,  and  demographic  characteristics,  are  deferred  to  future  studies. 

Implications  for  Future  Research 

While  this  was  intended  primarily  as  an  exploratory  study,  its  findings  suggest  several 
implications  for  future  diffusion  research.  Most  notably,  it  derived  a  set  of  factors  without  placing  a  priori 
restrictions  on  the  type  of  factor  being  sought.  In  contrast,  while  several  well-validated  frameworks  exist, 
most  previous  research  in  this  area  focuses  on  part  of  the  diffusion  problem.  Rogers  identifies  innovation 
characteristics  facilitating  adoption;  Ely  does  the  same  for  environmental  conditions;  Zaltman  &  Duncan 
look  chiefly  at  factors  hindering  adoption.  As  the  field  was  being  defined,  this  approach  made  a  great  deal  of 
sense,  amounting  to  a  divide  and  conquer  strategy.  Now,  perhaps,  a  more  appropriate  role  for  future  inquiry 
is  the  integration  of  these  perspectives:  looking  at  the  diffusion  problem  as  a  whole  (the  innovation  and  its 
environment)  from  whatever  mix  of  perspectives  (acceptance  and  resistance)  offers  the  most  rigorous 
explanation  of  the  observed  phenomena. 

Another  implication  is  embedded  in  the  method  employed  to  derive  critical  issues  from  the 
identified  factors.  The  technique  used  is  new,  as  far  as  the  author  is  aware.  It  has  the  advantages  of  being 
simple,  and  allowing  an  objective  means  of  comparing  factors  to  identify  the  most  cost-effective 
opportunities  for  intervention.  However,  while  this  procedure  may  be  intuitively  appealing,  and  appears  to 
have  produced  reasonable  results  in  this  circumstance,  it  has  not  been  validated  outside  this  study.  Future 
investigations  of  this  sort  may  wish  to  learn  if  it  has  wider  application. 

Finally,  the  findings  of  this  study  offer  an  in-depth  look  at  the  factors  affecting  the 
adoption/rejection  decision  for  the  full  range  of  participants  in  the  implementation  of  a  particular  training 
device  in  a  particular  Army  discipline.  While  this  evidence  is  probably  adequate  to  suggest  attention  to  the 
identified  factors  in  any  future  implementation,  it  is  not  sufficient  to  conclude  that  there  are  no  other  factors 
of  equal  importance  to  other  systems,  or  in  other  disciplines  or  environments.  Ideally,  further  expansions 
of  scope  should  be  conducted  until  no  new  issues  are  revealed.  Such  augmentation  would  have  the 
additional  benefit  of  clarifying  the  factors  already  discovered,  offering  insight  into  issues  such  as 
circumstances  causing  a  particular  factor  to  assume  prominence  in  a  particular  implementation, 
interrelationships  between  factors,  or  correlations  between  factors  and  demographic  characteristics.  The 
result  could  be  a  user's  guide  for  implementing  technological  support  for  any  training  in  any  environment. 


REFERENCES 

Berman,  P.  (1981),  Educational  Change:  An  Implementation  Paradigm,  in  R.  Lehming  and  M.  Kane 
(eds.),  Improving  Schools.  Beverly  Hills,  CA:  Sage  Publications. 

Bogdan,  R.  and  Biklen,  S.  (1992),  Qualitative  Research  for  Education.  Needham  Heights,  MA:  Allyn  and 
Bacon. 

Burkman,  E.  (1987),  Factors  Affecting  Utilization,  in  Robert  M.  Gagne  (ed.),  Instructional  Technology: 
Foundations  (pp.  429-455).  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates,  Publishers. 

Ely,  D.  (1990),  Conditions  that  Facilitate  the  Implementtion  of  Educational  Technology  Innovations, 
Journal  of  Research  on  Computing  in  Education,  23,2:298-305. 

Evans,  R.  (1968),  Resistance  to  Innovation  in  Higher  Education.  San  Francisco:  Jossey-Bass, 
Incorporated. 

Fullan,  M.  and  A.  Pomfret  (1977),  Research  on  Curriculum  and  Instruction  Implementation,  Review  of 
Educational  Research,  Winter,  1977. 


a 

ERIC 


15 


145 


Garland,  K.  (1991).  Diffusion  and  Adoption  of  Instructional  Technology.  In  Gary  J.  Anglin  (Ed.), 
Instructional  Technology:  Past,  Present,  and  Future  (pp.253-258).  Englewood,  CO:  Libraries 
Unlimited,  Incorporated. 

Kaufman,  R.  (1972).  Educational  System  Planning.  Englewood  Cliffs,  NJ:  Prentice-Hall,  Inc. 

Martin,  C.  (1988,  April).  Ethnographic  Methods  for  Studying  Microcomputer  Implementation  in 

Schools.  (Paper  presented  at  the  Annual  Meeting  of  the  American  Educational  Research  Association, 
New  Orleans,  LA.) 

Mosteller,  F.  (1981),  Innovation  and  Evaluation,  Science,  21 1 :881-886. 

Rogers,  E.  (1983),  Diffusion  of  Innovations  (Third  Edition).  New  York:  Free  Press. 

£ 

Rose,  S.  (1982),  Barriers  to  the  Use  of  Educational  Technologies  and  Recommendations  to  Promote  and 
Increase  Their  Use,  Educational  Technology,  22:12-15. 

Wellin,  E.  (1955),  Water  Boiling  in  a  Peruvian  Town,  in  Benjamin  D.  Paul  (ed.),  Health,  Culture,  and 
Community  (pp.  71-103).  New  York:  Russel  Sage  Foundation. 

Zaltman,  G.  &  R.  Duncan  (1977),  Strategies  for  Planned  Change.  New  York:  John  Wiley  and  Sons. 


James  B.  Ellsworth  is  with  the  United  States  Army  at  Fort  Huachuca,  Arizona.  This  paper  was  presented  at 
the  annual  meeting  of  the  Association  for  Educational  Communications  and  Technology  (AECT)  in 
February  1995.  The  author  would  like  to  thank  Dr.  Polly  Ulichny,  Ms.  Heather  Britt,  Ms.  Liz  White, 
Sister  Becky  James,  Ms.  Cappy  Smith,  and  Brother  John  Tan  for  their  valuable  comments  during  the 
original  study.  Correspondence  may  be  directed  to  James  B.  Ellsworth  at  Post  Office  Box  320,  Fort 
Huachuca,  AZ  8561 3-O320,  or  on  the  Internet  by  electronic  mail  to:  ELLSWORJ%HUAl  ©huachuca- 
emhl  l.army.mil 


ERIC 


146 


Title: 


Reflective  Self-Regulation  as  a  Facilitative  Factor 
in  Learning  from  Case-Based  Instruction 


Authors: 

Peggy  A.  Ertmer 
Purdue  University 
W.  Lafayette,  IN 


Timothy  J.  Newby 
Purdue  University 
W.  Lafayette,  IN 


Maureen  MacDougall 
Purdue  University 
W.  Lafayette,  IN 


a 

ERIC 


It 


147 


Abstract 


This  exploratory  study  integrated  information  from  quantitative  and  qualitative  data  sources  to  highlight 
how  different  students  responded  to,  and  learned  from,  case-based  instruction.  Fifty-eight  first-year 
veterinary  students,  enrolled  in  a  biochemistry  laboratory  course  which  utilized  case  studies  as  one  of  the 
primary  methods  of  instruction,  participated  in  the  study.  Quantitative  data,  gathered  from  students'  pre- 
course  performances  on  two  learning  inventories,  were  used  to  classify  students  according  to  their  initial 
levels  of  self-regulation.  By  identifying  those  students  who  fell  at  both  ends  of  the  inventory  scales, 
comparisons  could  be  made  between  different  types  of  learners.  Nine  students,  representing  high  and  low 
self-regulators,  were  interviewed  at  three  different  times  during  the  semester  to  explore  initial  and  changing 
responses  to  case-based  instruction.  Using  a  constant  comparative  method  of  anaylsis,  an  analytical 
framework  was  developed  which  identified  critical  factors  which  seemed  to  facilitate,  or  limit,  students' 
ability  to  learn  from  this  method. 
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Introduction 


Case-based  instruction  has  long  been  established  as  an  effective  teaching  method  in  schools  of  !aw  and 
business  and  is  increasingly  being  used  in  other  professional  fields  as  well,  including:  medicine,  political 
science,  journalism  (Knirk,  1991),  teacher  education  (Kagan,  1993;  J.  Shulman,  1992),  architecture  (Schon, 
1983,  1987),  educational  psychology  and  measurement  (Silverman,  Welty,  &  Lyon,  1992),  and 
instructional  design  (Romiszowski,  Mulder,  &  Pieter,  i990).  Although  case-based  instruction  is  not  new, 
J.  Shulman  (1992)  states  that  a  body  of  theory  and  research  has  yet  to  be  developed  which  explains  why 
"they  are  so  unusually  effective"  (p.  xv).  It  is  not  that  the  value  of  case  studies  has  not  been  debated; 
indeed,  there  is  a  large  literature  base  which  discusses  the  perceived  benefits  and  challenges  of  the  case 
method  of  instruction.  However,  when  Masoner  (1988)  recently  reviewed  any  literature  that  could  be 
construed  as  an  evaluation  of  :ase  methods,  he  found  mostly  "anecdotal  evidence,  unpublished  studies,  and  a 
small  assortment  of  unrelated  and  non-cumulative  published  studies"  (summarized  in  Kleinfeld,  1991 ,  p.  3). 
Furthermore,  the  little  research  that  has  been  done  in  this  area  has  mainly  been  concerned  with  the  relative 
impact  of  case-based  instruction  on  the  class  as  a  whole,  as  compared  to  other,  more  traditional  methods 
(lecture,  expository  texts),  and  has  been  almost  entirely  quantitative  in  nature  (Beckinan,  1972;  Masoner, 
1988;  L.  Shulman,  1992). 

To  date  very  little  work  has  been  done  which  carefully  examines  how  individual  learners  respond  to,  and/or 
learn  from,  case-based  methods  of  instruction  (Knirk,  1991).  The  possibility  that  individual  learning  needs 
may  not  "match  up"  with  the  specific  characteristics  and  demands  of  this  method  seems  to  be  downplayed,  if 
not  completely  ignored.  The  general  implication  in  the  literature  is  that  cases  are  more  motivating  for  all 
learners  that  they  promote  better  transfer  in  all  learners,  and  that  they  can  transform  all  learners  into  better 
problem  solvers  and  critical  thinkers  (e.g.,  L.  Shulman,  1992).  Only  a  few  educators  have  suggested  that 
case-based  instruction  does  not  "work"  for  all  learners  (Cossom,  1991 ;  Welty,  Silverman,  Lyons,  1990). 
Cossom  (1991,  p.  151)  states  that,  "clearly  (case-based  instruction)  is  not  a  teaching/learning  method  that 
appeals  to  all  students,  nor  is  it  one  that  draws  neutral  responses."  He  cites  Daloz  in  postulating  that 
"students  at  different  levels  of  moral  and  cognitive  development  will  have  varying  degrees  of  comfort  with 
ambiguity,  lack  of  a  "right"  response,  and  multiplicity  of  views"  (p.  150),  qualities  inherent  in  a  case-based 
approach. 

It  is  fairly  well-acknowledged  that  students  approach  learning  in  a  wide  variety  of  ways;  they  come  to 
school  with  different  amounts  and  kinds  of  background  knowledge;  they  have  reached  different  levels  of 
cognitive  development;  they  hold  varying  degrees  of  motivation  for  the  content  to  be  learned  (Paris  & 
Byrnes,  1989).  We  cannot  begin  to  assume  that,  left  to  their  own  devices,  all  students  will  respond 
similarly  to,  or  benefit  equally  from,  any  single  instructional  method,  including  case-based  instruction.  It 
would  be  beneficial  to  understand  how  students  learn  from  cases;  to  identify  specific  learner  characteristics 
and  particular  learning  strategies  that  facilitate,  or  limit,  learning  from  this  approach. 

The  three  main  purposes  of  this  study  were: 

1)  to  examine  students'  initial  responses  toward,  and  strategies  for  learning  from,  case-based^instruction 

2)  to  describe  how  those  responses  and  strategies  changed  over  the  course  of  a  semester  in  which  case- 

based  instruction  was  utilized 

3)  to  compare  the  initial  and  changing  responses  and  strategies  of  different  types  of  learners  as  they 

progressed  through  a  case-based  course 

This  study  is  based  on  the  assumption  that  responses  toward,  and  subsequent  performance  in,  a  case-based 
course  depends  highly  on  one's  level  of  self-regulation— defined  here  as  the  ability  (and  propensity)  to 
implement,  monitor,  and  evaluate  various  learning  strategies  for  the  express  purpose  of  facilitating 
knowledge  growth.  Although  highly  self-regulated  learners  may  have  no  prior  knowledge  of  the  specific 
content  being  taught  (e.g.,  biochemistry),  or  previous  experience  with  the  specific  instructional  methods 
being  used  (e.g.,  case  studies),  their  ability  to  activate,  alter,  and  sustain  appropriate  learning  practices 
predicts  that  they  would  achieve  academic  success  in  familiar,  as  well  as  new,  learning  contexts 
(Zimmerman,  1990).  By  observing  how  highly  self-regulated  students  approach  case-based  instruction, 
specific  facilitating  attributes,  attitudes,  and  approaches  might  be  identified.  Conversely,  by  observing  the 
manner  in  which  low  self-regulators  approach  case  studies,  we  might  pinpoint  potential  debilitating 
responses  and/or  ineffective  strategies  which  limit  learning.  Ultimately,  it  is  hoped  that  we  might  identify 
enabling  instructional  conditions  and  strategies  which  enhance  the  performance  of  all  kinds  of  learners  in  a 
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case-based  learning  environment.  It  is  hoped  that  information  gained  from  less  regulated  students  will 
indicate  not  just  the  limiting  factors  which  might  be  minimized,  but  also  the  supportive  factors  which 
might  be  maximized  in  subsequent  learning  situations.  By  examining  how  different  students'  responses  and 
strategies  promote,  or  limit,  their  ability  to  learn  from  cases,  instructional  methods  for  facilitating  and 
enhancing  case-based  learning,  for  all  students,  might  be  identified. 

The  assumption  that  self-regulation  skills  are  a  facilitative  factor  in  learning  from  any  instructional  method 
(including  case-based  instruction)  is  well-grounded  in  the  literature  (e.g.,  Lindner  &  Harris,  1993; 
Weinstein,  1988;  Zimmerman,  1990).  According  to  the  definition  above,  students  with  high  levels  of  self- 
regulation  possess  the  attributes  and  skills  that  would  be  likely  to  enhance  performance  in  a  case-based 
course.  Self-regulated  students  are  aware  of  important  learning  variables  and  possess  the  necessary 
knowledge  (of  self,  of  learning  strategies,  and  of  task  requirements)  to  take  conttol  of  their  own  learning 
environments  (Palincsar  &  Brown,  1989).  Furthermore,  they  are  motivated  to  do  so. 

However,  the  assumption  that  self-regulation  skills  can  be  enhanced  through  involvement  in  case-based 
instruction  is  also  suggested  by  research.  Although  students  with  low  levels  of  self-regulation  may  not 
have  the  skills  or  strategies  needed  to  perform  well  initially  in  this  environment,  specific  pedagogical 
features  of  case-based  instruction  are  thought  to  facilitate  the  development  of  these  skills.  For  example,  in 
a  review  of  programs  designed  to  teach  higher  order  cognitive  skills,  Resnick  noted  that  the  most  successful 
programs  were  those  which  employed  "cooperative  problem  solving  and  meaning-construction  activities" 
(summarized  in  Resnick  &  Klopfer,  1989,  p.  8).  Such  small  and  large  group  problem  solving  activities  are 
central  to  the  case  approach;  the  effective  case  teacher  operates  not  as  a  knowledge-dispensor  but  as  a  model , 
coach,  and  facilitator.  Furthermore,  case-based  discussions  provide  opportunities  for  each  student  to  serve  as 
a  monitor  and  reflector  for  the  others,  thus  facilitating  the  growth  of  metacognitive  monitoring  and 
reflection.  By  involving  students  in  the  "disciplined  and  productive  mental  work"  (p.  10)  of  case-based 
instruction,  it  is  believed  that  students  can  learn  how  to  learn. 

Based  on  the  assumptions  stated  above,  it  is  expected  that  the  following  phenomena  would  be  observed  in  a 
case-based  learning  environment: 

1 .  Initially,  high  self-regulated  students  would  be  more  comfortable  with  the  case  approach  and  would 
utilize  more  effective  strategies  to  analyze  cases  than  low  self-regulators. 

2.  Low  self-regulated  students'  performance  would  improve  over  time,  with  sustained  involvement 
and  continued  practice. 

How  can  we  gauge  the  progress  that  learners  make  in  regards  to  becoming  self-regulated?  Is  self-regulation 
a  characteristic  that  students  either  have  or  don't  have?  Or  is  self-regulation  a  skill,  likened,  perhaps,  to 
expertise,  that  evolves  slowly  over  time?  If  it  is  developmental,  how  can  we  assess  whether  students  are, 
indeed,  becoming  more  self-regulated  in  their  learning?  Unlike  many  learning  theorists  today,  Ridley 
(1990)  believes  that  all  behavior  is,  to  some  degree,  self-regulated.  Thus  Ridley  proposes  a  self-regulation 
continuum  which  ranges  from  "unreflectively  automatic  self-regulated"  on  the  low  end  to  "reflectively 
intentional  self-regulated"  on  the  high  end.  This  continuum  provides  a  model  for  describing  how  learners 
with  different  levels  of  self-regulation  might  respond  to  new,  and  potentially  uncomfortable,  academic 
situations.  Ridley's  continuum  also  offers  a  viable  means  for  gauging  changes  in  learners'  levels  of  self- 
regulation.  For  example,  if  students  become  less  automatic  and  more  thoughtful  in  their  approach  to  cases 
(i.e.,  they  make  some  movement  along  the  continuum),  this  might  be  a  reasonable  indicator  that  they  are 
making  progress  in  the  development  of  "reflectively  intentional"  self-regulation  habits. 

Theoretical  Framework 

This  exploratory  study  was  designed  to  seek  "interpretive  outcomes"  (Peshkin,  1993),  that  is,  insights  that 
might  begin  to  clarify  the  complex  processes  involved  in  students'  learning  from  case-based  instruction.  To 
this  end,  both  qualitative  and  quantitative  methods  were  used.  Quantitative  methods,  nested  within  a  larger 
interpretive,  or  qualitative,  framework,  were  employed  primarily  as  a  means  to  classify  students  according  to 
their  initial  levels  of  self-regulation.  An  overarching  interpretive  framework  (e.g.,  Eisner,  1991;  Erickson, 
1986)  guided  the  search  for  patterns  of  responses  to  case-based  instruction.  According  to  Dillon  ( 1989), 
qualitative  research  has  been  used  "as  a  means  of  answering  previously  unanswered  questions  concerning 
how  . .  students  learn"  (p.  227).  In  this  study,  interpretive  inquiry  offered  a  realistic  means  for  more  clearly 
defining  how  different  students  respond  to,  and  learn  from,  case-based  instruction. 
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Students'  perceptions  of  learning  in  a  case-based  course  were  examined  with  the  express  purpose  of 
describing  case-based  instruction  from  the  point  of  view  of  the  students.  Phenomenology  provided  a  useful 
theoretical  framework  for  exploring  students'  responses  to,  and  learning  from,  this  type  of  instruction.  Kuh 
(1993)  states  that  "it  is  impossible  to  understand  the  human  experience  without  taking  into  account  the 
complicated,  mutually  shaping  events,  actions,  and  motivations  of  the  individual . .  .  under  study"  (p.  278). 
This  particular  theoretical  framework  enables  a  researcher  to  tap  into  individual  perceptions  regarding 
learning  from  cases  using  the  techniques  of  questioning,  focusing,  reflecting,  and  intuiting  (Van  Manen, 
1990).  In  this  way,  the  structures  of  meaning  embedded  in  students'  experiences  of  case-based  instruction 
could  be  explored. 

Methods 


Description  of  the  site,  course,  and  participants 

The  site.  A  professional  school  of  veterinary  medicine,  located  at  a  large  midwestern  university,  is  one 
of  only  27,  nationwide,  which  grants  the  degree  of  Doctor  of  Veterinary  Medicine  PVM).  The  four  year 
program  typically  limits  enrollrr  ;nt  to  60  students  per  class.  The  school  catalog  states  that  "each 
prospective  student  is  required  tu  complete  a  prescribed  preprofessional  curriculum  for  two  or  more 
collegiate  years  before  admission  to  the  school."  Other  criteria  for  admission  (e.g.,  academic  performance, 
maturity,  and  motivation),  once  considered  to  be  quite  stringent,  have  been  relaxed  in  recent  years  to  boost 
declining  enrollments,  resulting  in  a  more  heterogeneous  student  population. 

The  course  A  required  freshman  course,  "Systemic  Physiology  II  (Biochemistry),"  provided  the 
context  for  this  study.  The  course  (VPH  442)  is  described  in  the  syllabus  as  "an  introduction  to  biomedical 
principles  and  their  application  to  veterinary  medicine."  The  lab  portion  of  the  course  focused  on  the 
application  of  principles  learned  in  lecture  to  hypothetical  paper-patients  who  mimicked  real-life  disorders. 
Two  sections  of  the  lab  were  offered  by  the  same  instructor  during  the  spring  semester  in  which  the  study 
took  place.  Although  students  were  specifically  assigned  to  one  of  the  two  labs,  they  typically  attended  the 
lab  which  was  most  convenient  for  them  during  any  specific  week.  Therefore,  the  total  number  of  students 
in  any  one  lab  fluctuated  between  15  and  45  students. 

VPH  442  was  designed  so  that  students  received  approximately  two  hours  of  both  lab  and  lecture  each 
week  throughout  the  16  week  semester.  Biochemical  case  studies  were  used  as  one  of  the  primary  methods 
of  instruction  and  as  an  evaluation  tool  in  both  lab  sections  of  the  course.  Case  study  presentations 
followed  a  fairly  typical  pattern:  students  were  presented  with  a  limited  description  of  an  animal  in  distress. 
Some  of  the  animal's  symptoms  were  described,  accompanied  by  appropriate  illustrations  and  diagrams,  and 
then  students  were  asked  to  analyze  the  patient's  condition  and  to  make  tentative  diagnoses  and 
recommendations  for  action.  Students  worked  in  groups  and  were  encouraged  to  ask  questions,  to  check 
available  resources,  and  to  consult  with  the  instructor  before  arriving  at  preliminary  conclusions.  All  case 
investigations  concluded  with  a  large  group  discussion  in  which  student  recommendations  were  considered 
in  light  of  supporting  (or  negating)  lab  and  clinical  data. 

Participants.  Students:  Sixty-one  first-year  veterinarian  students  were  enrolled  in  the  biochemistry  lab 
course  during  the  time  of  the  study.  Data  sets  from  three  students  were  incomplete,  leaving  a  total  of  58 
students  who  completed  the  study.  Of  these  58  participants,  66%  were  female  (n  =  38).  Ages  ranged  from 
20  to  40  years  (M  =  24.22,  £D  =  3.99).  Although  the  majority  of  students  (n  =  36)  had  completed  a 
bachelor  of  science  degree  prior  to  entering  this  program,  their  levels  oi  education  ranged  from  two  years  of 
post-secondary  education  (n  =  15)  to  a  masters  degree.  Most  students  in  this  program  had  a  background  in 
either  biological  science  or  agriculture,  although  other  fields  were  also  represented.  In  addition,  several 
students  had  previous  job  experience  in  fields  as  diverse  as  accounting,  management,  and  dentistry. 

The  teacher.  Marie  McDonnell  (not  her  real  name)  is  a  34-year-old  practicing  veterinarian,  as  well  as  a 
graduate  student  in  the  School  of  Education.  Besides  working  1/2  time  at  a  small  animal  clinic  at  the  time 
of  the  study  Marie  also  had  a  graduate  teaching  assistantship  which  required  her  to  spend  approximately  20 
hrs/week  designing  and  teaching  the  biochemistry  lab.  Six  of  the  other  8  core  teachers  in  the  freshmen 
curriculum  were  also  DVMs,  but  only  2  had  ever  practiced.  This  put  Marie  in  a  unique  position  within  the 
school-  although  she  was  technically  a  student,  she  had  more  practical  experience  than  most  of  the 
professors.  Marie  had  been  teaching  this  particular  lab  for  two  years,  gradually  increasing  her  use  of  case 
studies  until  they  had  come  to  represent  the  primary  instructional  method.  According  to  previous  course 
evaluations,  students  liked  the  case  approach. 
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Data  Sources 

Information  from  quantitative  and  qualitative  data  sources  was  integrated  to  highlight  how  different  students 
responded  to,  and  learned  from,  case-based  instruction.  Quantitative  data  were  gathered  from  students' 
performances  on  two  individual  learning  inventories  administered  at  the  beginning  of  the  semester,  as  well 
as  from  GPAs  and  final  course  grades.  These  data  sources  provided  the  means  to  classify  students  so  as  to 
compare  the  responses  of  different  types  of  learners  to  case-based  instruction.  Qualitative  data  were  gathered 
from  periodic  student  interviews  and  three  written  case  analyses  with  additional  evidence  provided  from 
classroom  observations,  teacher  handouts,  and  course  evaluation  forms.  These  data  sources  were 
instrumental  in  understanding  the  responses  of  these  different  learners  as  they  participated  in  a  case-based 
biochemistry  laboratory.  In  addition,  all  data  gathered  at  the  beginning,  middle,  and  end  of  the  semester 
were  compared  to  identify  how  students'  responses  and  strategies  changed  over  the  course  of  the  semester. 

Measurement  Tools 

Two  self-report  learning  inventories,  based  in  the  self-regulation  literature  (MSLQ  and  SRLI),  were  used  to 
assess  the  strategies  students  used  to  facilitate  their  learning.  These  strategies  included:  cognitive  (rehearsal, 
organization,  and  elaboration),  metacognitive  (monitoring,  evaluating,  and  reflecting  over  one's  learning), 
motivational  (interest,  value,  and  confidence  for  learning  specific  course  content),  contextual  (awareness  of 
task  requirements),  and  environmental  control  (time  and  resource  management). 

The  Motivated  Strategies  for  Learning  Questionnaire  (MSLQ;  Pintrich,  Smith,  Garcia,  &  McKeachie, 
1991)  is  a  self-report  questionnaire  designed  to  assess  college  students'  motivational  beliefs  and  their  use  of 
different  learning  strategies  for  a  specific  college  course.  The  81  item,  Likert-type  survey  is  divided  into  15 
subscales  that  ask  students  to  respond  to  items  regarding  motivation,  cognitive  and  metacognitive  strategy 
use,  and  management  of  outside  resources,  using  a  7  point  response  format  (1  =  not  at  all  true  of  me;  7  = 
very  true  of  me).  The  motivation  section  consists  of  six  subscales  (31  items)  that  assess  students'  goals 
and  value  beliefs  for  a  course,  beliefs  about  their  skill  to  succeed,  and  anxiety  about  tests.  The  cognitive 
and  metacognitive  strategy  section  includes  five  subscales  (31  items)  assessing  student  use  of  cognitive  and 
metacognitive  strategies  such  as  rehearsal,  elaboration,  and  organization.  The  resource  management  section 
includes  four  subscales  (19  items)  concerning  management  of  outside  resources. 

The  second  measurement  tool,  the  Self-Regulated  Learning  Inventory.  (SRLI)  (Lindner,  &  Harris,  1992)  is 
comprised  of  75  self-report  items  categorized  into  five  subscales  including:  Metacognition,  Learning 
Strategies,  Motivation,  Contextual  Sensitivity,  and  Environment  Control.  The  items  are  presented 
randomly  on  a  single  test  using  a  5  point  Likert  format. 

Data  collection  and  analyses 

Quantitative.  All  students  were  administered  the  two  self-report  learning  inventories  during  the  first 
week  of  the  1994  spring  semester.  The  results  of  these  inventories  were  used  to  classify  students  into  one 
of  three  categories  representing  each  student's  level  of  self-regulatory  skills:  high,  medium,  or  low. 
Because  it  is  difficult  to  capture  the  dynamics  of  self-regulated  learning  with  a  self-report  instrument  (P. 
Pintrich,  informal  presentation,  November  16,  1993)  it  was  considered  advantageous  to  use  two  separate 
measures  of  self-regulated  learning  with  each  serving  as  a  check  on  the  other.  Students  who  scored 
relatively  high  (or  low)  on  both  scales  might  be  more  reliably  classified  as  high  (or  low)  self-regulators.  In 
an  effort  to  achieve  a  more  observable  contrast  between  different  types  of  students,  those  who  fell  within  the 
"medium"  category  were  not  evaluated  further  in  this  study,  although  they  participated  in  the  same  class 
activities  and  completed  the  same  course  assignments  as  the  other  students. 

Qualitative.  In  order  to  gain  an  understanding  of  students'  perceptions  of  their  learning  experiences  in  a 
case-based  course,  it  was  essential  to  utilize  methods  that  tapped  those  perceptions.  Thus, 
phenomenological  interviews  (Kuh,  1993,  p.  278)  "whereby  the  inquirer  gains  access  to  the  meanings 
individuals  attach  to  their  own  experience(s)  using  a  semi-structured  interview  guide"  constituted  one  of  two 
primary  data  sources.  In  order  to  hear  individual,  as  well  as  collective  voices,  four  students  were  selected, 
purposively  (Patton,  1990),  from  both  the  high  and  the  low  self-regulated  groups  to  be  interviewed  at  three 
different  times  during  the  semester.  Purposive  sampling  helped  to  ensure  that  representative  perceptions 
would  be  obtained  regarding  learning  from  case-based  instruction.  One  additional  student  (Roslyn),  who  fell 
within  the  "middle"  range  of  self-regulators,  was  also  included  in  the  interview  sample  as  it  was  expected 
that  her  perspective,  as  an  "older"  student  with  many  years  of  clinical  experience,  may  be  particularly 
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informative.  Although  Roslyn  achieved  the  5th  highest  score  on  the  MSLQ,  her  score  was  average  on  the 
SRLI.  For  all  subsequent  analyses,  this  student's  data  were  combined  with  that  of  the  high  self-regulators. 

Phenomenological  interviews  were  used  to  assess  students'  responses  toward  the  case  method  as  well  as 
their  strategies  for  learning  from  this  type  of  instruction.  Interviews  were  semi-structured,  beginning  with 
open  ended  statements  ("Tell  me  about  Biochemistry  Lab."),  and  concluding  with  a  few  direct  questions 
("What  do  you  do  when  you  are  given  a  case  study  to  analyze?";  "How  do  you  feel  when  you  are  asked  to 
analyze  a  case  study?").  Based  on  the  work  of  Pintrich  and  his  colleagues  (e.g.,  Garcia  &  Pintrich,  1994; 
Pintrich  et  al.,  1991)  which  suggests  that  students'  performances  are  differentially  affected  by  the 
motivational  subcomponents  of  interest/enjoyment  ("How  interesting  is  this  instruction  to  you?";  "How  do 
you  like  this  type  of  instruction?"),  value/importance  ("How  valuable  is  th's  instruction  in  terms  of  your 
future  work?"),  and  efficacy  ("How  confident  do  you  feel  learning  from  this  type  of  instruction?"),  additional 
questions  addressed  these  components. 

Students'  written  case  analyses  constituted  the  second  primary  data  source.  During  the  3rd,  8th,  and  16th 
weeks  of  the  semester,  all  students  were  given  a  written  case  study  and  asked  to  list:  1)  the  potential 
problems  or  issues  presented  in  the  case,  2)  prioritized  differential  diagnoses,  and  3)  possible  ways  of 
handling  the  case.  Besides  a  simple  count  of  the  number  of  problems  and  differential  diagnoses  listed,  the 
content  of  students'  written  responses  were  examined  to  identify  strategies  employed  by  students'  in  their 
case  analyses.  The  first  level  of  coding  identified  the  number  and  variety  of  cognitive  strategies  (e.g., 
underlining,  highlighting,  mnemonics,  etc.)  used  by  each  student.  Students'  analyses  were  then  searched  for 
any  indications  of  the  use  of  metacognitive  strategies,  particularly  those  indicative  of  reflecting  on, 
monitoring,  and  evaluating  one's  own  work  (e.g.,  self-questioning,  using  tentative  words,  writing  notes  to 
oneself,  etc.). 

Over  the  course  of  the  semester,  27  interviews  were  conducted  and  63  written  case  analyses  were  examined 
(3  analyses  completed  by  each  of  the  lOhighand  1 1  low  classified  students).  All  of  the  interview  and  case 
responses  were  read  and  coded,  using  a  constant  comparative  method  of  analysis  (Glaser  &  Strauss,  1967). 
Reliability  was  established  by  having  an  independent  reviewer  classify  a  random  sample  (20%)  of  interview 
statements  using  the  coding  criteria  described  in  the  next  section.  Judgments  of  students'  analyses,  in  terms 
of  accuracy  and  thoroughness,  were  made  by  the  instructor. 

By  reading  and  rereading  interview  transcriptions  and  case  analyses  and  then  comparing  findings  across  data 
sources,  across  time,  and  across  students,  an  analysis  framework  was  gradually  developed  and  refined. 
Beginning  with  transcriptions  of  the  first  set  of  interviews,  a  search  for  significant  meanings  was 
undertaken.  As  students'  first  case  analyses  were  examined,  key  linkages  were  sought  to  connect  similar, 
emerging  themes.  In  addition,  attention  was  paid  to  negative  cases  which  suggested  opposing  explanations. 
Subsequent  readings  of  initial  interviews  and  case  analyses,  combined  with  readings  of  additional  interviews 
and  analyses,  facilitated  the  gradual  development  of  an  interpretive  model  of  students'  responses  to,  and 
strategies  for  learning  from,  case-based  instruction.  As  similar  patterns  began  to  emerge  from  each  main 
data  source,  secondary  data  sources  were  examined  (e.g.,  field  notes  from  classroom  observations,  teacher 
case  documents,  and  student  course  evaluations)  in  an  effort  to  cross-validate  assertions.  Finally,  an 
analytic  framework  was  created  which  integrated  evidence  from  all  sources. 

Results  and  Discussion 

This  section  of  the  paper  discusses  results  from  both  quantitative  and  qualitative  data  sources  with  the  main 
focus  being  on  results  related  to  the  defined  purposes  of  this  study.  After  addressing  the  first  two  purposes 
regarding  initial  and  changing  responses  toward,  and  strategies  for  learning  from,  case-based  instruction,  an 
analysis  framework  is  presented  as  a  means  of  interpreting  these  resuits.  Due  to  space  limitations,  an 
abbreviated  version  of  some  results  is  provided;  a  more  detailed  explanation  is  available  from  the  first 
author. 

Quantitative 

Initial  analysis  of  students'  scores  on  the  two  learning  inventories  identified  9  high  and  1 1  low  self- 
regulators.  Selection  ci  iterion  required  students'  scores  on  both  scale  s  to  be  greater  than  .5  standard 
deviations  above,  or  below,  the  mean.  In  addition,  students  chosen  to  be  interviewed  were  those  who  had 
scores  on  the  metacognitive  and  self-regulation  subscales  (Self-Regulation  Subscale-MSLQ;  Metacognitive 


ERIC 


153 


t.i 


Subscale-SRLI)  that  were  greater  than,  or  equal  to,  .5  standard  deviations  above,  or  below,  the  class  mean. 
As  mentioned  previously,  an  additional  student  from  the  "middle"  group  was  included  with  the  9  high  self- 
regulators  for  subsequent  analyses. 

Selection  criterion  resulted  in  the  following  sample  of  high  and  low  students:  All  high  students  were 
female;  4  of  the  1 1  low  students  were  female.  The  average  age  of  the  high  students  was  25.2  years 
compared  to  an  average  age  of  23.6  years  for  the  low  students.  Average  GPAs  were  3.186  for  the  high 
group;  3.107  for  the  low  group.  Table  1  presents  a  compilation  of  specific  demographic  information  for 
students  in  each  of  the  high  and  low  groups,  as  well  as  their  scores  on  the  two  learning  inventories.  Group 
averages  for  the  high  and  low  students  are  also  included.  Although  all  names  are  pseudonyms,  the 
demographic  information  is  accurate.  Results  from  additional  descriptive  and  correlational  analyses  of  the 
survey  data  are  not  reported  here,  however,  descriptive  analyses  based  on  student  interview  or  case-analyses 
data  are  integrated  within  the  qualitative  results. 


Table  1 

Demographic  Information  for  Students  Classified  as  High  and  Low  Self-Regulators 


Group 
(H;L) 

Student 

Sex 

Age 

MSLQ 
Score 

SRLI 
Score 

Yrsof 
school 
(post  HS) 

#  of  Prior 
Courses 

GPA 

Related  Experience 

H* 

Marci 

F 

22 

486 

333 

3 

1 

4.00 

BS-Biology;  worked  4  yrs  in 
small  animal  clinic 

H* 

Winnie 

F 

26 

495 

317 

5 

0 

2.52 

BS-Biology; 
Dressage  instructor 

H* 

Sharon 

F 

24 

478 

304 

4 

2 

2.94 

BS-Animal  Science;  5  yrs  with 
small  animal  vet;  2  yrs  equine 
emer.  referral  service 

H 

Deidra 

F 

23 

446 

287 

4 

1 

3.08 

No  experience  listed 

H* 

Mallry 

F 

23 

450 

318 

4 

3 

3.61 

Pre-Vet;  worked  in  Animal 
Control 

H 

Tami 

F 

27 

451 

334 

5 

1 

3. 16 

BS-ZooIogy 

T  T 

H 

Lucy 

r 

A  £T 

402 

Z/O 

4 

i 
1 

1  A  1 

BS-Industrial  Mngmnt; 
Pre-Vet 

H 

Cynr.y 

F 

27 

443 

293 

9 

1 

3.46 

MS-Zoology 

H 

Liiyja 

F 

21 

462 

289 

2 

2 

2.86 

Pre-Vet;  Equestrian  Team 

H** 

Roslyn 

F 

32 

467 

272 

4 

1 

2.80 

13  yrs  in  Vet  Clinic;  3.5  yrs  in 
small  animal  ICU 

H-Ave 

All 
F 

25 

463 

302.5 

4.4 

1.3 

3.18 

L 

Jon 

M 

23 

369 

241 

4  j 

2 

2.80 

BS-Animal  Science 

L* 

Ronald 

M 

23 

340 

227 

4 

0 

3. 

BS-Biology;  Raises  snakes 

L 

Mick 

M 

25 

395 

228 

6 

2 

3.63 

BS-Accounting 

L 

Ira 

F 

25 

360 

196 

2 

1 

3.17 

AS-Science 

L 

Candy 

F 

26 

389 

230 

4 

2 

3.18 

BS-Biology 

L* 

Deena 

F 

22 

367 

233 

4 

2 

2.71 

BS-Animal  Bioscience; 
Pre-Vet 

L 

Carl 

M 

22 

388 

247 

4 

1 

3.08 

BS-Biology;  Pre-Vet 

L* 

Chrissy 

F 

22 

396 

239 

4 

1 

3.53 

BS-Eng/Chem  Double  Major 

L 

Matt 

M 

25 

382 

229 

3 

1 

3.26 

Biology  Club 

L* 

George 

M 

21 

379 

215 

2 

0 

2.60 

Pre-Vet;  Swine,  cattle 
experience 

L 

Marvin 

M 

26 

363 

246 

4 

2 

3.16 

BS-Medical  Technology 

L-Ave 

4F 

24 

375.3 

230.1 

3.72 

1.27 

3.11 

*  Students  who  were  interviewed 

**  Student  who  scored  at  high/med  level;  included  with  high  students  for  all  analyses 
MSLQ  M  =  415.25,  SD  =  35.42;  SRLI  M  =  264.55,  SD  =  29.51 
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Qualitative — Responses  toward  case-based  instruction 

Initial  and  changing  responses  toward  case-based  instruction  were  tapped  by  asking  probing  questions  during 
three  successive  interviews  which  were  specifically  related  to:  1)  students'  interest  in  using  cases,  2)  the 
perceived  value  of  using  cases,  and  3)  students'  confidence  for  learning  from  this  method.  Students'  initial 
and  subsequent  responses  to  these  questions  are  summarized  below,  along  with  a  comparison  between 
responses  made  by  high  and  low  students,  when  possible. 

Initial  responses.  Interest:  In  describing  their  initial  reactions  to  the  use  of  cases  in  the  biochemistry 
lab,  almost  all  students  interviewed  (both  high  and  low)  indicated  that  they  thought  that  cases  would  make 
the  class  more  interesting  and  more  fun.  Some  likened  it  to  a  game,  or  a  puzzle,  mentioning  the 
challenging,  enjoyable  aspects  of  cases.  Others  described  how  their  motivation  was  positively  affected  and 
how  it  had  spilled  over  into  other  coursework.  Most  students  simply  indicated  that  case-based  instruction 
was  more  interesting  than  their  other  classes  and  provided  a  nice  change  of  pace.  Only  Deena  qualified  her 
comments  by  stating,  "They're  good,  but  only  to  a  certain  degree." 

Value:  All  9  students  claimed  that  cases  were  "real-life"  and  had  some  practical  benefits.  Although  3 
of  the  4  low  students  noted  that  cases  would  help  them  remember  more,  they  judged  that  this  would  not 
affect  other  coursework  or  career  goals.  This  stands  in  contrast  to  the  5  high  students  who  all  mentioned 
that  the  case  method  was  very  valuable  to  their  future  careers,  as  well  as  to  other  coursework.  Not  only  did 
these  high  students  value  the  practicality  of  case-based  instruction,  but  they  also  noted  some  global  benefits 
such  as  learning  the  "problem-solving"  approach  and  integrating  their  knowledge. 

Confidence:  All  of  the  students  expressed  some  concern  regarding  their  ability,  at  this  point  in  their 
careers,  to  accurately  diagnose  the  cases  they  were  given  ("Well,  I  have  no  idea  what  it  could  possibly  be 
because  I  only  know  2  diseases  and  that's  all!").  They  used  words  such  as:  scared,  frustrated,  nervous,  and 
intimidated.  However,  all  but  one  low  student  mentioned  that  this  lack  of  knowledge  would  result  in 
increased  effort  ("I  probably  put  more  effort  into  understanding  what  we  learn  in  this  class  because ...  I 
know  that  it  will  definitely  be  useful.").  Additionally,  everyone  but  Roslyn  (who  had  a  lot  of  previous 
experience  in  a  vet  clinic)  expressed  feelings  of  discouragement  due  to  a  lack  of  knowledge  that  they  judged 
was  essential  to  "solve"  the  cases. 

Changing  responses  Interest:  Even  though  some  low  students  were  "burned  out"  by  the  end  of  the 
semester,  most  of  their  frustration  seemed  to  be  related  to  outside  sources  (other  tests,  deadlines,  etc.),  rather 
than  to  the  course  itself.  Low  students  seemed  to  be  especially  sensitive  to  these  other  factors.  Course 
evaluations  support  the  conclusion  that  most  students  still  enjoyed  the  course  at  the  end  of  the  semester. 
On  a  scale  from  1-10  the  average  course  rating  was  8.9.  Unsolicited  comments  mentioned  enjoying  the 
case-study  approach  and  having  fun  in  the  class:  "Truly  enjoyed  labs.  They  made  my  others  relevant." ;  "I 
greatly  enjoyed  the  lab!!!  It  was  the  only  class  in  our  freshman  curriculum  which  makes  you  think  logically 
about  cases  you  will  see  as  a  clinician." 

Value:  As  the  semester  went  on,  there  seemed  to  be  a  shift  in  emphasis  regarding  which  aspect  of  the 
case  approach  was  valued  most.  Low  students,  particularly,  moved  from  a  focus  on  practical  benefits 
(change  of  pace,  ability  to  remember  more  facts)  to  more  overarching  benefits  (application  of  knowledge, 
learning  the  problem-solving  approach).  The  process,  rather  than  the  product,  began  to  take  on  increased 
value. 

Confidence:  As  students  became  more  comfortable  with  the  problem-solving  approach,  as  their 
knowledge  base  increased,  and  as  their  experience  with  cases  increased,  they  became  more  confident  of  their 
case  analyses.  However,  the  primary  factor  influencing  students'  confidence  levels  seemed  to  be  the  amount 
of  prior  knowledge  and  previous  experience  they  had.  "If  I  had  a  broader  repertoire  of  possibilities,  I  would 
have  felt  more  confident."  However,  some  high  students  actually  became  motivated  by  their  lack  of 
knowledge:  "It's  like  a  kid  with  with  a  new  video  game!" 

Students  from  both  groups  seemed  to  redefine  success  during  the  semester  and  adjusted  their  judgments 
of  confidence  to  match.  Students  began  to  emphasize  "coming  close"  rather  than  naming  a  specific  disease 
that  may  have  caused  the  presenting  problems.  If  diagnoses  were  "in  the  ballpark"  students  judged  that  they 
had  been  successful  ("I  knew  this,  this,  and  this,  but  being  able  to  list  one  specific  problem,  no,  I  don't 
know  enough  diseases  or  technical  names  to  write  anything  down.  But  it  comes  close."). 

By  Time  3,  "scared"  and  "nervous"  feelings  were  no  longer  mentioned,  yet  all  four  low  students  became 
particularly  frustrated  by  their  lack  of  knowledge,  by  the  specific  case,  or  by  the  tediousness  of  the  work. 
These  students  were  more  apt  to  complain  about  other  responsibilities,  other  course  requirements,  and  other 
external  factors  (time  and  length  of  lab)  that  contributed  to  their  levels  of  stress. 

Summary  of  Initial  and  Changing  Responses.  When  the  initial  and  changing  responses  of  high  and 
low  students  are  compared,  an  interesting  difference  is  noted.  For  the  most  part,  high  students  started  with, 


and  maintained,  a  positive  attitude  toward  the  use  of  case  studies  throughout  the  semester.  Sensitivities 
were  noted  only  in  response  to  the  type  of  case  being  analyzed.  In  contrast,  low  students  started  with  a 
rather  narrow  view  of  the  value  of  cases  and  were  less  confident  of  their  initial  analysis  skills.  By  mid- 
semester,  however,  most  of  these  students  had  gained  in  confidence  and  had  broadened  their  view  of  the 
value  of  case-based  instruction.  Yet  by  the  end  of  the  semester,  other  pressures  appeared  to  overcome  some 
of  these  students  and  their  motivation  ai.d  confidence  for  learning  from  this  method  decreased.  It  is 
encouraging  to  note,  however,  that  two  of  these  students  (George,  and  to  some  extent,  Chrissy),  continued 
to  make  progress  in  becoming  more  comfortable  with,  and  strategic  in,  their  approach  to  the  case  method. 

Qualitative— Strategies  used  in  case-based  learning 

Students'  initial  and  changing  strategies  for  learning  from  cases  were  garnered  from  two  sources:  1)  written 
case  analyses  and  2)  students'  responses  to  interview  questions  regarding  what  they  did  when  they  were 
given  a  case.  Both  cognitive  (e.g.,  underlining,  using  mnemonics,  organizing  information)  and 
metacognitive  (e.g.,  writing  notes  or  parenthetical  comments  to  oneself,  asking  rhetorical  questions,  using 
tentative  words)  strategy  use  were  examined.  Although  a  detailed  explanation  of  this  aspect  of  the  study  is 
not  reported  here,  Table  2  outlines  the  different  types  of  strategies  that  were  used  by  high  and  low  students 
in  each  of  their  case  analyses. 


Table  2 

Number  of  Students  Using  Cognitive  and  Metacognitive  Strategies  on  Each  Case  Analysis 


Cognitive  Strategy 

Case  1 

Case  2 

Case  3 

Underlines,  Circles,  Highlights 
High/Low 

8/6 

5/6 

8/6 

Uses  Indicators 
High/Low 

6/1 

4/4 

2/0 

Organizes  Information 
High/Low 

0/1 

6/7 

2/0 

Uses  Mnemonics 
High/Low 

3/3 

3/3 

2/1 

Summarizes 

High/Low 

0/0 

1/5 

1/1 

Metacognitive  Strategy 

Case  1 

Case  2 

Case  3 

Asks  Rhetorical  Questions 
High/Low 

4/4 

3/5 

8/8 

Writes  Notes  to  Self 
High/Low 

7/5 

5/5 

6/0 

Uses  Tentative  Words 
High/Low 

8/7 

4/7 

3/4 

Looks  for  Links 
High/Low 

7/5 

5/4 

3/2 

Uses  Evidence 
High/Low 

9/10 

2/3 

4/5 

Considers  Context 
High/Low 

3/1 

1/1 

8/7 

Looks  at  Big  Picture 
High/Low 

2/2 

2/0 

1/0 

Uses  "I"  Statements 
High/Low 

5/5 

1/1 

7/4 

Uses  Technical  Words 
High/Low 

7/6 

9/11 

8/8 

Is  Complete  and  Thorough 
High/Low 

2/0 

7/3 

2/4 
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Analysis  Framework. 

A  search  for  commonalties  across  cases  revealed  two  general  orientations  to  case-based  instruction,  which 
were  loosely  coded  as  limiting  and  facilitative.  Based  on  a  modification  of  Ridley's  (1990)  description  of  a 
self-regulated  learning  continuum  (from  automatic  to  purposeful),  interview  statements  and  case-analyses 
strategies  were  identified  which  seemed  to  represent  both  ends  of  this  continuum.  Statements  of  an 
"automatic"  nature  were  considered  to  be  indicative  of  a  limiting  approach  to  case-based  learning,  whereas 
statements  of  a  more  "purposeful"  nature  were  thought  to  represent  a  facilitative  approach. 

Limiting  factors,  identified  from  interview  comments,  included  such  things  as:  setting  product  goals  for  the 
course  ("I'd  like  to  basically  keep  up  with  the  course"),  using  automatic  or  habitual  strategies  with  very 
little  thought  ("That's  how  I've  always  done  it"),  holding  a  narrow  view  of  what  constitutes  professional 
knowledge  and  expertise  ("I  felt  nervous  about  doing  the  case  and  also  what  the  teacher  might  think  if  I 
wrote  the  totally  wrong  thing"),  excusing  one's  performance  because  of  limited  resources  ("I  wasn't  given 
enough  information"),  and  being  highly  sensitive  to  contextual  and  environmental  influences  ("Classes  tend 
to  run  over  time,  and  it's  kinda  hard  to  get  motivated  to  do  much  of  anything  except  go  home"). 

Facilitating  factors,  identified  from  interview  statements,  consisted  of:  setting  process  goals  for  the  course 
("I'd  like  to  learn  as  much  as  possible  how  to  approach  cases"),  being  aware  of  how  one  learns  ("I  like  to  be 
able  to  see  something,  to  visualize  the  problem"),  recognizing  one's  automatic  responses  to  novel  and/or 
difficult  learning  situations,  and  then  implementing  alternative  strategies,  if  necessary,  in  order  to  be 
successful  ("You  gotta  tell  yourself  that  you're  not  a  senior"),  holding  a  wide  view  of  the  meaning  of 
knowledge  and  expertise  ("The  first  time  I  tried  to  get  too  specific  instead  of  doing  a  more  general  approach 
to  it"),  and  drawing  on  available  personal  resources  in  order  to  master  a  valued  learning  goal  ("I  try  to  think 
of  what  we'd  done  in  other  classes;  things  I  learned  before  that  could  help  me").  Besides  prior  knowledge 
and  experience,  personal  resources  also  included  facilitating  factors  such  as:  openness  to  emotional 
challenges,  persistence  at  difficult  tasks,  and  increased  or  sustained  motivation  ("You  want  to  try  because 
it's  like  playing  a  game  to  see  if  you  can  win"). 

Interview  transcriptions  were  searched  thoroughly  for  specific  comments  which  seemed  to  represent  either  of 
these  orientations  to  case-based  instruction.  These  comments  were  then  classified  according  to  the 
components  of  the  facilitative-limiting  framework  described  above  and  depicted  in  Table  3  (i.e.,  process  vs 
product  goals;  reflective  vs  automatic  responses;  wide  vs  narrow  lens;  professional  vs  student  view;  hardly 
or  highly  sensitive  to  contextual  factors;  and  reference  to  available  vs  limited  resources).  A  total  of  480 
comments  were  classified  into  these  6  categories.  To  ensure  that  consistency  was  maintained  within 
categories,  a  second  judge  classified  a  random  selection  of  comments  (20%)  from  each  category.  The 
following  reliabilities  were  reported  for  each  category:  Process/Product  =  90%;  Reflective/Automatic  = 
86%;  Wide/Narrow  Lens  =  81%;  Professional/Student  =  92%;  Contextual  Sensitivity  =  100%; 
Available/Limited  Resources  =  93%,  for  an  overall  interrater  reliability  of  90%.  Disagreements  were 
resolved,  in  most  cases,  by  providing  additional  contextual  information  for  specific  comments  or  by 
discussing  discrepancies  and  then  making  a  mutual  decision  regarding  final  classification. 
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Table  3 

Students'  Approaches  to  Case-Based  Instruction 


Facilitative 

Limiting 

Type  of  Goals 

Process 

•Learn  the  approach 
•Integrate  information 
•Gain  global  knowledge 

Product 

•Pass  the  course 

•Get  a  grade 

•Learn  the  right  answers 

Level  of 
Self-Awareness 

Reflective 

•Plans — thinks  then  acts 

•Monitors — adjusts  actions 

•Evaluates — approach  &  product 

Automatic 

•First  reactions 

•Habit 

•Unaware  of  own  learning  habits 

Range  of  Lens 

Wide 

•Values  general  diagnosis 
"Values  shared  expertise 
•Criteria  for  success  self-imposed 

Narrow 

•Values  specific  answer 
•Acquiesces  to  authority 
•Criteria  imposed  by  others 

Point  of  View 

Professional 

•Open  to  emotional  challenges 

•Willing  to  change 

Student 

•Self-protection 

•Survival  over  development 

Contextual  Sensitivity 

Not  sensitive  to: 
•Knowing  the  "answer" 

•Task  difficulty 

•Environmental  factors 

•Other  pressures 

Sensitive  (to: 
•Getting  a  grade 

•Type  of  case 

•Time  of  day,  length  of  case 
•Other  pressures 

Resources 

Available 
•Prior  knowledge 

•Previous  experience 

•Increased  motivation  and  effort 

Limited 

•Knowledge 

•Experience 
•Ability 

Qualitative — Comparison  of  responses  and  strategies  of  different  types  of  learners 
In  order  to  compare  the  initial  and  changing  responses  and  strategies  of  different  types  of  learners  as  they 
progressed  through  the  course  (purpose  #3),  student  interview  responses  were  mapped  onto  the  framework 
presented  in  Table  3  and  toen  compared  across  time  and  across  groups.  Table  4  presents  a  separate  matrix 
for  each  of  the  three  case  anal,  ses  in  which  each  student's  case  approach  is  defined,  in  terms  of  its  facilitative 
and  limiting  features.  Tins  is  followed  by  a  more  general  matrix  in  Table  5  which  summarizes  the 
information  from  the  first  three  matrices.  Each  matrix  was  completed  by  coding  students'  interview 
comments  in  terms  of  the  previously  identified  categories. 
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Table  4 

Interview  Content  Analvsis-Time  1 


Student 

Classification 
(high  vs  low) 

Process  vs 

Product 

Goals 

Reflective  vs 

Automatic 

Responses 

Wide  vs 
Narrow  Lens 

Professional  vs 
Student  View 

Contextual 
Sensitivity 

Available  vs 

Limited 

Resources 

Sharon 

H 

Process 

Balanced 

NEI* 

Professional 

Insensitive 

Available 

Marci 

H 

Balanced 

Balanced 

Balanced 

Professional 

rsfci 

Limited 

Mallry 

H 

Process 

Balanced 

wioe 

r  roicoSluudl 

NanntiirA 
OtJIlSlUVC 

Winnie 

H 

Process 

Balanced 

Balanced 

Professional 

Sensitive 

Balanced 

Roslyn 

H 

Balanced 

Balanced 

Wide 

Balanced 

NEI 

Available 

H-Gp 

Process 

Balanced 

Balanced 

Professional 

Mixed 

Available 

Deena 

L 

Product 

Automatic 

Narrow 

Layman 

Sensitive 

Limited 

Chrissy 

L 

Product 

Balanced 

NEI 

Layman 

Sensitive 

Balanced 

Ronald 

L 

Product 

Balanced 

Narrow 

Layman 

Sensitive 

Limited 

George 

L 

Balanced 

Reflective 

Narrow 

Professional 

Sensitive 

Limited 

L-Gp 

Product 

Mixed  Narrow 

Layman          I  Sensitive 

Limited 

*NEI=Not  Enough  Information  Available 


Interview  Content  Analvsis-Time  2 


Student 

Classification 
(high  vs  low) 

Process  vs 

Product 

Goals 

Reflective  vs 

Automatic 

Responses 

Wide  vs 
Narrow  Lens 

Professional  vs 
Student  View 

Contextual 
Sensitivity 

Available  vs 

Limited 

Resources 

Sharon 

H 

Balanced 

Reflective 

Wide 

Professional 

Sensitive 

Balanced 

Marci 

H 

Balanced 

Reflective 

Wide 

Professional 

Insensitive 

Available 

Mallry 

H 

NEI 

Automatic 

Wide 

Professional 

Insensitive 

Balanced 

Winnie 

H 

Process 

Reflective 

Balanced 

Professional 

Insensitive 

Limited 

Roslyn 

H 

Balanced 

Reflective 

Wide 

NEI 

Insensitive 

Available 

H-Gp 

Balanced 

Reflective 

Wide 

Professional 

Insensitive 

Available 

Deena 

L 

NEI 

Automatic 

Narrow 

NEI 

Sensitive 

NEI 

Chrissy 

L 

Product 

Reflective 

Narrow 

Professional 

Sensitive 

Limited 

Ronald 

L 

Process 

Balanced 

Narrow 

Balanced 

Insensitive 

Balanced 

George 

L 

Process 

Reflective 

Balanced 

Professional 

Insensitive 

Available 

L-Gp 

!  Process 

Reflective 

Narrow 

Professional 

Mixed 

I  Mixed 

1   

*NEI=Not  Enough  Information  Available 
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Interview  Content  Analysis-Time  3 


Student 

Classification 

Process  vs 

Reflective  vs 

Wide  vs 

Professional  vs 

Contextual 

Available  vs 

(high  vs  low) 

Product 

Automatic 

Narrow  Lens 

Student  View 

Sensitivity 

Limited 

Goals 

Responses 

Resources 

Sharon 

H 

Process 

Reflective 

Wide 

Professional 

Balanced 

Available 

Marci 

H 

Process 

Reflective 

Wide 

Professional 

Insensitive 

Balanced 

Mallry 

H 

Process 

Balanced 

Wide 

Professional 

Insensitive 

Limited 

AXAinnip 
VT  11  u  lie 

l  X 

iTOCCSo 

Keuccuve 

WKjc 

Professional 

Insensitive 

Limited 

Roslyj 

H 

Process 

Reflective 

Wide 

Professional 

Insensitive 

Available 

H-Gp 

Process 

Reflective 

Wide 

Professional 

Insensitive 

Mixed 

Deena 

L 

Process 

Automatic 

N£I 

NEI 

Sensitive 

NEI 

Chrissy 

L 

Product 

Reflective 

Narrow 

Balanced 

NEI 

Balanced 

RonaJd 

L 

Balanced 

Automatic 

Balanced 

Layman 

Sensitive 

Limited 

George 

L 

Process 

Reflective 

NEI 

Professional 

Sensitive 

Balanced 

L-Gp 

Process 

Mixed 

Mixed 

Mixed 

Sensitive 

Balanced 

*NEI=Not  Enough  Information  Available 


To  complete  the  coding  process,  a  number  of  decisions  were  necessary.  For  example,  students  were  credited 
with  using  a  certain  approach  even  if  only  one  representative  statement  was  made  during  an  entire  interview. 
If  both  facilitative  and  limiting  statements  were  made  regarding  a  single  category  (e.g.,  goals,  range  of 
view,  etc.),  students  were  classified  as  having  a  "balanced"  approach,  even  though  the  number  of  statements 
representing  each  type  of  approach  may  not  have  been  equal.  Based  on  the  rationale  that  even  one  statement 
indicated  use  of  a  particular  approach,  it  was  judged  important  to  acknowledge  even  isolated  uses  of 
facilitative  or  limiting  strategies.  Furthermore,  it  would  not  be  expected  that  students  would  make  radical 
changes  in  their  learning  orientations  in  a  short  time  period,  but  would  change  overtime  as  new  strategies 
gradually  replaced  old,  less  effective  ones.  Thus,  it  is  likely  we  would  see  an  overlap  in  orientations  for  a 
period  of  time  as  well  as  some  occasional  regression  to  more  familiar  patterns  under  stressful  conditions. 
Additionally,  change  would  be  expected  to  be  uneven;  that  is,  some  categories  would  show  changes  before 
others.  The  intent  of  these  summary  matrices  was  simply  to  get  a  rough  idea  of  how  students'  approaches 
to  case-based  instruction  may  have  changed  over  the  course  of  the  semester. 

Group  comparisons.  Besides  defining  each  student's  approach  in  the  categories  presented,  Table  4  also 
include  a  "group"  entry  which  indicates  the  most  common  approach  used  by  students  in  each  of  the  high  and 
low  groups.  Group  entries  were  determined  using  simple  guidelines.  If  more  than  half  the  students  in  the 
gTOup  used  a  particular  approach,  the  group  entry  names  that  approach  as  representative  of  the  group.  If 
responses  did  not  cluster  within  a  category,  the  word  "mixed"  is  used,  indicating  more  variation  among 
group  members.  The  "balanced"  rating,  used  previously  to  indicate  that  both  approaches  had  been  used  by  a 
student,  was  most  often  used  as  evidence  to  support  a  facilitative  group  entry.  Although  "summarizing"  the 
approaches  of  individuals  is  not  always  easy  or  necessary,  it  was  attempted  here  in  order  to  make  a 
comparison  between  different  types  of  learners  and  their  approaches  to  case-based  instruction.  As  such, 
these  group  entries  provide  some  insight  into  the  similarities  and  differences  between  groups  at  three 
different  times  in  the  semester.  By  comparing  these  summaries  across  time,  we  begin  to  see  how  groups, 
as  well  as  individuals,  changed  from  the  beginning  to  the  end  of  the  semester. 

Summary  matrix.  A  final  summary  matrix  (See  Table  5)  was  created  by  assigning  a  point  value  to 
each  approach,  so  that  a  ready  comparison  could  be  made  across  time  and  across  students.  The  use  of  a 
facilitative  approach  (e.g.,  process  goals,  reflective  responses,  wide  lens,  etc.)  was  assigned  2  points;  a 
balanced  approach  received  1  point,  and  a  limiting  approach  (e.g.,  product  goals,  automatic  responses, 
narrow  lens,  etc.)  received  0  points.  This  table  allows  us  to  see,  at  a  glance,  where  the  different  groups  of 
students  started,  and  where  they  ended,  the  semester.  For  example,  by  the  end  of  the  semester  all  of  the 
high  students  had  adopted  process  goals,  held  professional  views  of  learning  from  cases,  and  had  broadened 
their  conception  of  knowledge  to  include  respecting  others'  opinions  and  valuing  general,  as  opposed  to 
specific,  answers.  One  area  that  showed  little  change  was  that  of  accessing  available  personal  resources. 
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Some  high  students  still  expressed  concern  over  their  lack  of  knowledge  and  seemed  unable  to  draw  on 
previous  experiences. 


Table  5 

interview  Content  Analysis-Summary  of  Time  1-2-3 

(Facilitative  approach  =  2  pts.;  Balanced  approach  =  1  pt.;  Limiting  approach  =  0  pts.) 


Student 

Classification 
(high  vs  low) 

Process  vs 

Product 

Goals 

Reflective  vs 
Automatic 
Res  nonces 

1  Wt}  UV/l  BOW  o 

Wide  vs 
Narrow  Lens 

Professional  vs 
oiuueni  v  icw 

Contextual 

Qpncitivi  t\t 

ocridiLiviiy 

Available  vs 
Resources 

Sharon 

H 

2—1—2 

1—2—2 

x*— 2— 2 

2—2—2 

2—0—1 

2—1—2 

Marci 

H 

1—1—2 

1—2—2 

l_2—2 

2—2—2 

x— 2— 2 

0—2—1 

Mallry 

H 

2— x— 2 

1—0—1 

2—2—2 

2—2—2 

1—2—2 

1—1—0 

Winnie 

H 

2—2—2 

1—2—2 

1—1—2 

2—2—2 

0—2—2 

1—0—0 

Roslyn 

H 

1—1—2 

1—2—2 

2—2—2 

I— x— 2 

x— 2— 2 

2—2—2 

Summary 

1.6-1.3- 
2 

1—1.6-1.8 

1.5-1.8-2 

l.g—2— 2 

1—1.6—1.8 

1.2-1.2-1 

Deena 

L 

0— x— 2 

0— 0— 0 

0— 0— x 

0— x— x 

0— 0— 0 

0— x— x 

Chrissy 

L 

0— 0— 0 

1—2—2 

x— 0— 0 

0—2—1 

0— 0— x 

1—0—1 

Ronald 

L 

0—2—1 

1—1—0 

0—0—1 

0—1—0 

0—2—0 

0—1—0 

George 

L 

1—2—2 

2—2—2 

0— 1— x 

2—2—2 

0—2—0 

0—2—1 

Summary 

.25-1.3-1.3  1  1—1.3—1 

0— .3— .5       .5—1.3—1      |  0—1—0       |  .3— 1— .7 

*x=Not  Enough  Information  Available 


The  pattern  of  change  for  low  students  was  different  from  that  of  high  students.  Whereas  the  high  students 
seemed  to  make  fairly  steady  progress  toward  the  use  of  facilitative  strategies,  low  students  demonstrated 
much  more  back  and  forth  movement.  Overall,  the  low  students  demonstrated  a  limiting  approach  to  the 
first  case,  yet  described  using  a  more  facilitative  approach  for  the  second  case.  There  are  a  number  of 
possible  reasons  for  this  difference  including  changing  orientations,  format  of  the  case  (length  of  case, 
species  involved,  questions  asked),  increased  competency  with  specific  analysis  procedures,  and  semester 
timing.  At  the  time  of  the  third  interview,  however,  some  low  students  were  observed  to  rely  on 
automatic,  familiar  responses  rather  than  employing  these  more  facilitative  strategies.  Again,  there  are  a 
number  of  possible  reasons  including  increased  pressures  from  external  sources  (assignment  deadlines, 
impending  final  exams)  and  the  specific  type  of  case  presented.  This  third  case  was  formatted  somewhat 
differently  and  may  have  frustrated  students  who  had  been  learning  to  analyze  cases  by  applying  a  standard 
procedure.  By  providing  much  more  information  about  the  context  of  the  problem  (the  barnyard  setting) 
but  no  lab  results,  students  had  to  ask  more  questions  and  determine  additional  tests  needed  before  they  could 
complete  their  analyses.  This  additional  demand,  at  this  time  in  the  semester,  may  have  shaken  students' 
confidence  and  increased  their  frustration  causing  them  to  rely  on  more  familiar  strategies.  Having  to  deal 
with  a  number  of  new  factors  at  one  time  may  have  caused  these  students  to  return  to  what  was  familiar  as  a 
way  of  preventing  cognitive,  or  stress,  overload. 

As  a  group,  low  students  made  promising  gains  in  terms  of  the  goals  they  established  for  this  case-based 
course.  Three  out  of  the  four  low  students  included  some  mention  of  process  goals  in  their  second  and  third 
interviews,  while  only  one  student  had  started  the  semester  with  a  process  goal.  This  in  itself  is 
noteworthy  and  suggests  that  case-based  instruction  may  be  beneficial  in  helping  students  focus  more  on  the 
learning  process  as  opposed  to  the  products  of  learning  (grades,  facts,  task  completion). 

The  individual  learners.  Table  5  also  enables  us  to  see  how  particular  individuals  responded  to  the  case 
approach  and  to  identify  characteristics  of  those  who  appeared  to  make,  or  not  make,  gains  along  the  self- 
regulated  continuum.  A  closer  look  at  these  discrepant  cases  may  point  to  specific  factors  which  may  have 
facilitated  or  limited  learning  for  these  particular  learners.  I  provide  a  brief  view  of  just  a  few  students  to 
illustrate  the  range  of  differences  noted. 
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Like  the  other  low  students,  Ronald  started  out  with  a  fairly  limited  case  approach,  yet  had  showed  progress 
in  every  area,  but  one,  by  the  time  of  the  2nd  interview.  However,  Ronald  became  so  flustered  with  the 
third  case  that  his  performance  suffered  in  almost  every  area.  He  recognized  that  having  a  systematic 
approach  was  important  but  admitted  that  he  had  not  been  able  to  be  systematic  under  conditions  which 
included  an  unfamiliar  animal,  time  constraints,  and  a  different  case  format.  It  is  encouraging  to  note, 
however,  that  Ronald  did  not  regress  to  pre-course  levels  in  every  area.  He  still  mentioned  some  process 
goals  and  satisfaction  with  achieving  a  general  diagnosis.  Given  a  reduction  of  other  stresses,  Ronald  still 
judged  that  he  could  be  successful  in  future  case  analyses.  "If  I  didn't  have  any  major  tasks  pending  I  can 
probably  do  one  now  very  confidently  provided  I  had  the  resources." 

Although  Deena  came  to  recognize  the  value  of  the  case  process,  she  demonstrated  very  little  change  in  any 
of  the  other  areas.  Deena  was  never  completely  sold  on  the  case  method,  but  more  importantly,  was  never 
thrilled  about  being  part  of  this  study.  Although  Deena  expressed  initial  interest  and  motivation  for  the 
course,  this  was  quickly  lost  when  outside  pressures  mounted.  During  the  first  interview  Deena  indicated 
that  the  case  method  may  actually  hurt  her  study  habits  as  she  would  not  have  as  many  notes  from  which  to 
study.  She  valued  traditional  teaching  methods  (lecture  and  textbook  assignments)  and  wished  that  the 
teacher  would  include  some  of  these  ("I  don't  think  that  you  can  learn  just  from  cases.  Pure  lecture  is 
important  in  clarifying  certain  concepts").  This  particular  response  is  not  unexpected.  In  the  veterinarian 
educational  literature,  Herron,  Wolf,  and  DiBrito  (1990)  suggest  that  studenls  may  resist  a  problem-solving 
approach  if  they  judge  that  it  will  require  more  effort  and/or  time  to  achieve  the  same  results  as  those 
achieved  through  traditional  didactic  approaches.  According  to  Ridley  (1990),  how  a  student  responds  to 
new  academic  demands,  in  terms  of  feeling  either  challenged  or  stressed,  depends  on  that  student's  particular 
orientation  to  self-regulation.  In  Deena's  case,  her  "limiting"  approach,  as  defined  here,  seems  to  have  added 
to  her  feelings  of  being  stressed. 

Deena  took  the  opportunity  during  the  2nd  and  3rd  interviews  to  voice  her  frustration  and  anger  over  having 
to  do  extra  work  for  this  study.  Because  the  interviews  required  even  more  of  her  time,  these  were  typically 
much  shorter  than  with  other  students,  and  some  useful  information  was  not  obtained.  It  is  probably  fair  to 
say  that  we  still  do  not  know  how  Deena  learned  from  cases,  only  how  she  responded  to  case-based 
instruction  given  these  particular  circumstances.  What  we  have  learned  from  Deena's  comments  is  how 
important  it  is  to  make  course  assignments  relevant  and  meaningful  (students  did  not  get  credit  for  these 
cases),  and  to  provide  individual  feedback  after  each  (students  only  participated  in  a  general  discussion  of 
each  case  following  their  analyses).  Deena's  criticisms  can  help  us  identify  those  aspects  of  a  case-based 
learning  experience  which  may  be  most  meaningful  to  the  participants. 

Roslyn's  experiences  in  this  course  stand  in  sharp  contrast  to  those  of  Deena.  Roslyn's  prior  clinical  work, 
as  well  as  her  strong  facilitative  attitude,  made  case-based  learning  a  natural  for  her.  Despite  suffering  from 
frustrations  with  other  courses,  a  lack  of  knowledge  about  specific  types  of  cases,  and  a  certain  amount  of 
fatigue  at  the  end  of  the  day,  Roslyn  never  lost  sight  of  the  value  of  the  case  method  for  her  career.  She 
recognized  her  limitations  as  a  learner  and  initiated  strategies  to  overcome  them  ("My  initial  response  was, 
'Oh  my,  it's  a  large  animal  again.'  But  I  just  said,  'Use  the  steps'  and  so  that's  what  I  did.").  Roslyn 
seemed  to  make  continual  movement  forward  along  the  self-regulated  continuum;  once  a  facilitative 
approach  was  adopted,  there  was  never  any  indication  that  it  had  been  set  aside. 

Sharon  started  out  as  one  of  the  strongest  learners.  Her  initial  approach  included  powerful  strategies  such  as 
trying  to  see  the  big  picture  and  trying  to  relate  new  information  to  old.  As  luck  would  have  it,  the  first 
case  study  involved  problems  that  Sharon  was  somewhat  familiar  with.  Her  confidence  and  motivation 
were  high  and  her  analysis  was  strong.  Although  Sharon's  interview  comments  continued  to  indicate  great 
value  in  the  method,  she  struggled  as  other  pressures  began  to  accumulate  and  when  later  cases  were  not  as 
familiar.  Sharon  demonstrated  remarkable  persistence  during  our  interviews  but  her  last  two  case  analyses 
were  weak  and  sketchy.  It  is  interesting  to  speculate  that  perhaps  Sharon  is  our  best  example  of  what  a 
self-regulated  learner  does  when  pressures  become  overwhelming.  Sharon's  motivation  and  persistence  in 
the  course  remained  high;  her  performance  on  these  two  optional  case  analyses,  which  provided  no  credit  or 
individual  instructor  feedback,  received  much  less  attention.  Sharon  chose  to  use  her  time  and  effort  to 
perform  well  where  it  was  most  important  to  do  so. 

General  Discussion.  These  examples  illustrate  only  a  few  of  the  many  unique  experiences  that  students 
had  in  this  case-based  course.  What  can  we  learn  from  such  a  wide  range  of  responses?  What  general 
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factors,  if  any,  can  be  seen  operating  throughout?  Of  the  six  facilitating  and  limiting  variables  detailed 
earlier,  the  one  that  seems  consistently  influential  is  the  type  of  goals  students  set  for  themselves  in  this 
particular  course.  This  finding  supports  other  research  in  the  area  of  self-regulation  which  indicates  that 
learning  and  performance  goals  may  exert  different  effects  on  self-regulatory  activities  (reported  in  Schunk, 
1994).  In  this  study,  students  who  focused  on  learning  the  approach  tended  not  to  get  as  frustrated  by  their 
lack  of  knowledge  or  by  not  achieving  a  single  "right"  answer.  Every  new  case  gave  them  the  opportunity 
to  practice  their  approach  and  to  refine  their  skills.  Students  who  adopted  process  goals  for  themselves  (as 
opposed  to  just  recognizing  the  value  of  the  approach,  as  in  Deena's  case)  remained  interested,  motivated, 
and  confident  in  their  ability  to  learn  in  this  course. 

On  the  other  hand,  those  who  adopted  product  goals  were  mainly  concerned  with  learning  more  of  the  facts 
which  they  judged  would  help  them  problem-solve  in  the  future.  These  students  seemed  more  easily 
frustrated,  restless,  and  less  confident  in  their  skills.  Their  overall  experience  in  the  course  was  tempered  by 
this  concern  for  doing  better,  knowing  more,  and  appearing  competent.  This  seemingly  negative  influence 
was  most  noticeable,  not  so  much  in  terms  of  case  analyses  performances  or  course  grades,  but  moreso  in 
students'  levels  of  comfort  with,  and  interest  in,  the  course  as  a  whole  and  in  the  specific  tasks  at  hand. 

It  is  interesting  to  note  that  neither  initial  levels  of  self-regulation,  nor  number  and  types  of  strategies  used, 
were  highly  related  to  teacher  ratings  of  students'  case  analyses.  There  are  many  possible  explanations  for 
this  including:  students'  lack  of  motivation  for  completing  these  specific  cases;  inability,  on  our  part,  to 
adequately  discern  students'  use  of  cognitive  and  metacognitive  strategies;  varying  levels  of  students'  prior 
knowledge;  and  teacher's  use  of  grading  criteria  which  focused  on  accuracy  of  diagnoses  (product  goal)  as 
well  as  analysis  procedures  (process  goal).  Zimmerman  (1994)  has  suggested  that  perceived  value  plays  a 
critical  role  in  one's  willingness  to  self-regulate.  In  this  study,  students'  approaches  appeared  to  be  highly 
influenced  by  the  perceived  value  of  the  course,  its  methods,  and  its  specific  tasks.  Whereas  Deena  saw 
some  value  in  the  course,  she  saw  little  value  in  the  method  and  no  value  in  the  specific  tasks  she  was 
asked  to  complete.  Other  students  may  not  have  valued  the  specific  tasks,  but  their  strong  value  for  the 
course  and  its  methods  appeared  influential  in  maintaining  their  motivation  and  effort  throughout  the 
semester.  A  high  value  on  the  case  approach,  as  mentioned  above,  seemed  especially  influential  in  students' 
development  and/or  use  of  case  analysis  and  self-regulation  skills. 

Conclusions  and  Implications 

Although  case-based  instruction  is  widely  heralded  as  being  a  powerful  teaching  method,  Sykes  and  Bird 
( 1992)  lament  that  these  is  no  research  literature  that  explores  the  nature  of  learning  through  cases.  One  of 
the  goals  of  this  study  was  to  further  our  understanding  of  how  students  learn  from  case-based  instruction. 
By  examining  students'  initial  and  changing  responses  toward,  and  strategies  for  learning  from,  case-based 
instruction  it  was  anticipated  that  specific  learner  and  instructional  factors  which  facilitate,  or  limit,  case- 
based  learning  might  be  identified.  By  comparing  the  responses  and  strategies  of  different  types  of  learners, 
defined  by  levels  of  self-regulation,  a  wide  range  of  responses  could  be  illustrated.  Students  who  represented 
opposing  ends  of  the  self-regulation  continuum  would  be  expected,  at  least  initially ,  to  demonstrate  different 
levels  of  comfort  with,  and  strategies  for  learning  from,  a  case  approach.  Examining  changes  in  these 
students'  responses  and  strategies  over  the  course  of  a  semester  provided  the  means  to  investigate  the 
assumption,  stated  earlier,  that  the  case  method  of  instruction  might  provide  a  viable  means  for  fostering 
"reflectively  intentional"  self-regulation  in  low-regulated  learners. 

As  expected,  high  self-regulated  learners  initially  demonstrated  more  comfort  with  the  case  approach,  as  well 
as  more  powerful  strategies  in  analyzing  cases,  than  low  self-regulators.  However,  initial  strategy  use  was 
commonly  tied  to  established  patterns  of  strategy  use,  which  for  high  self-regulators,  tended  to  include 
higher  level  strategies,  such  as  organizing  and  elaborating  on  information.  Comfort  seemed  to  be  related, 
primarily,  to  the  type  of  course  goals  students  set.  By  focusing  on  process  goals,  high  self-regulated 
students  emphasized  the  strategies  needed  to  analyze  a  case,  as  opposed  to  the  facts  needed  to  make  a  correct 
diagnosis.  Schunk  (1994)  indicates  that  process  goals  enable  students  to  experience  a  sense  of  self-efficacy 
for  skill  improvement  and  tend  to  engage  them  in  activities  that  enhance  learning  (e.g.,  effort  expenditure, 
persistence,  use  of  effective  strategies).  As  students  made  progress  in  their  case-analysis  approach,  their 
interest,  motivation,  and  confidence  increased,  or  was  sustained,  for  future  case  analyses. 
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In  contrast,  low  self-regulators  started  the  course  with  product  goals  which  focused  their  attention  on  getting 
a  grade,  completing  a  task,  or  learning  specific  facts.  These  types  of  goals  did  not  allow  these  students  to 
consider  the  importance  of  the  processes  or  strategies  underlying  successful  task  completion.  StudeUs 
seemed  more  concerned  with  getting  the  "right"  answer,  pleasing  the  teacher,  or  appearing  competent. 
Schunk  (1994)  suggests  that  product  goals  may  not  result  in  a  sense  of  self-efficacy  for  learning,  and  thus 
can  not  sustain  self-regulation.  However,  as  the  course  progressed,  promising  changes  in  students' 
orientations  were  noted.  By  the  time  of  the  second  i  nterview,  these  students  were  beginning  to  adopt 
process  goals,  to  be  more  reflective,  and  to  utilize  a  more  professional  outlook.  If  the  study  had  ended  at 
this  point,  we  might  have  judged  that  the  case  method  had  successfully  fostered  the  development  of  these 
specific  self-regulation  skills. 

What  happened  in  the  remaining  weeks  of  the  semester,  however,  highlights  the  importance  of  contextual 
factors  in  facilitating,  or  limiting,  students'  use  of  these  particular  self-regulation  strategies.  Although  there 
has  been  a  great  deal  of  previous  research  (cf.,  Weinstein,  1988)  which  has  demonstrated  that  self-regulatory 
processes  can  be  enhanced  through  instructional  intervention,  it  has  not  been  successfully  demonstrated  that 
these  changes  are  sustained  over  time  (Schunk,  1994).  Results  from  this  study  suggest  that  outside 
pressures  in  students'  lives  may  increase  their  sensitivity  to  other  instructional  factors  (type  of  case,  time  of 
day,  length  of  lab)  and  impede  their  use  of  self-regulation  skills.  This  finding  indicates  that,  although  the 
case  method  of  instruction  may  hold  promise  for  facilitating  the  development  of  self-regulation  in  low- 
regulated  students,  the  potential  influence  of  contextual  factors  needs  to  be  examined.  In  addition,  the 
interaction  between  learner  and  contextual  factors  is  an  area  which  could  benefit  from  future  research. 

By  being  aware  of  the  effect  that  contextual  factors  may  have  on  students'  use  of  self-regulatory  strategies, 
teachers  may  be  able  to  alter,  or  eliminate,  potentially  troublesome  factors  before  they  are  encountered.  For 
example,  teachers  can  help  students  establish  process  learning  goals  at  the  beginning  of  a  course  by 
emphasizing  the  strategies  underlying  successful  case  analysis.  By  basing  grading  criteria  on  students' 
ability  to  demonstrate  use  of  specific  learning  strategies  rather  than  bottom  line  diagnoses,  teachers  can  shift 
students'  emphasis  to  more  process-oriented  goals.  Furthermore,  if  teachers  can  give  students  more  choices 
in  terms  of  types  of  assignments,  types  of  cases  to  be  analyzed  (make  small  and  large  animal  cases 
available),  flexible  due  dates,  working  individually  or  in  groups,  accessing  available  resources,  etc.,  students 
may  be  more  capable  of  using  self-regulation  strategies.  At  the  very  least,  teachers  can  verbally 
acknowledge  the  pressures  students  are  encountering  and  thus  raise  levels  of  awareness  (as  a  first  step  in  the 
self-regulation  process)  and  establish  a  supportive  learning  environment.  Following  this,  teachei s  might 
model  their  own  strategies  for  dealing  with  similar  pressures. 

Lindner  and  Harris  (1993,  p.  4)  lament  that  "whether  and  under  what  conditions  learners  will  be  self- 
regulating  turns  out  to  be  a  complicated  matter."  By  utilizing  an  interpretive  approach  with  a  small  number 
of  students,  this  study  was  able  to  examine,  in  depth,  how  high  and  low  self-regulated  leainers  responded  to 
a  specific  instructional  environment.  An  analysis  of  students'  interview  responses  and  case  analysis 
strategies  enabled  us  to  identify  some  of  those  conditions  which  might  have  facilitated,  or  limited,  students' 
use  of  self-regulated  learning  strategies.  The  critical  factors  which  seemed  to  be  operating  in  this  study  were 
presented  in  the  form  of  a  facilitative-limiting  framework  outlining  two  general  orientations  to  case-based 
instruction.  Within  this  framework,  additional  factors  were  identified  which  seemed  particularly  influential 
in  determining  students'  overall  experience  in  this  case-based  course.  Future  research  should  be  directed 
toward  verifying  the  structure,  as  well  as  further  defining  the  individual  components,  of  this  framework. 
Following  refinement  and  verification,  this  framework  may  provide  other  researchers  and  educators  with  a 
viable  means  for  charting  students'  progress  in  the  development  of  self-regulation  skill.-,.  Given  the  current 
high  interest  among  school  practitioners  in  fostering  self-regulation  among  students  (Schunk,  1994), 
techniques  for  initiating,  supporting,  and  gauging  the  development  of  self-regulations  skills  will  be  both 
useful  and  necessary. 
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Introduction 


In  1992,  the  Georgia  Research  Alliance  released  a  "Request  for  Proposals"  (RFP)  for  establishing  an 
infrastructure  for  research  and  development  focused  on  advanced  telecommunication  technologies  and 
applications  in  the  state  of  Georgia.  A  group  of  faculty  (including  the  authors  of  this  paper)  in  the  College 
of  Education  at  The  University  of  Georgia  responded  to  this  RFP  and  obtained  funding  for  a  new  R&D 
facility  called  the  Learning  and  Performance  Support  Laboratory  (LPSL).  The  four  are  as  of  emphasis  in  the 
R&D  efforts  of  the  LPSL  are  1)  interactive  learning  environments,  2)  electronic  performance  support 
systems,  3)  alternative  performance  assessment  systems,  and  4)  information  access  systems. 

The  LPSL  is  committed  to  knowledge  acquisition,  theory  construction,  and  the  highest  ideals  of  applied 
research  and  development,  especially  as  they  relate  to  education,  training,  and  performance  problems  that 
must  be  solved  to  ensure  the  economic  viability  of  state  of  Georgia.  Advanced  telecommunications,  along 
with  genetic  engineering  and  environmental  technologies,  have  been  identified  as  the  three  key  industries  for 
Georgia  in  the  21  st  Century.  The  LPSL  is  committed  to  partnerships  and  collaborations  with  other 
institutions,  businesses,  and  agencies  in  the  state.  Current  partners  include  R&D  labs  and  centers  at  the 
five  institutions  (Georgia  Institute  of  Technology,  Georgia  State  University,  Emory  University,  Clark- 
Atlanta  University,  and  the  Medical  College  of  Georgia)  which  together  with  The  University  of  Georgia 
(UGA)  form  the  Georgia  Research  Alliance  (GRA).  Equally  important  partners  are  other  Georgia  public 
institutions  (e.g.,  colleges,  schools,  technical  training  centers,  etc.),  businesses  and  industries  (e.g.  AT&T, 
BellSouth,  and  CNN),  and  government  agencies  (e.g.,  the  U.  S.  Army  Signal  School  at  Ft.  Gordon).  The 
LPSL  is  also  establishing  substantive  collaborations  with  partners  from  around  the  USA  and  the  rest  of  the 
world.  For  example,  the  LPSL  has  established  a  relationship  with  the  "Cooperative  Multimedia  Centre" 
(CMC)  in  Perth,  a  collaboration  involving  four  universities  in  Western  Australia. 


Usability  Testing 

To  achieve  its  R  &  D  goals,  faculty  associated  with  the  LPSL  realized  that  they  would  need  state-of-the-art 
research  equipment.  One  of  the  primary  needs  was  obtaining  a  portable  usability  lab  for  software  testing. 
According  to  Shneiderman  (1987),  usability  is  a  combination  of  the  following  user-oriented  characteristics: 
1)  ease  of  learning,  2)  high  speed  of  user  task  performance,  3)  low  user  error  rate,  4)  subjective  user 
satisfaction,  and  5)  user  retention  over  time.  Hix  and  Harison  (1993)  and  Nielsen  (1993)  provide  guidance 
to  evaluating  user  interface  issues,  a  process  known  as  usability  testing.  Usability  testing  is  especially 
critical  in  the  design,  dissemination,  and  implementation  of  interactive  multimedia  for  education,  training, 
performance  support,  and  information  access  (cf.,  Blattner  &  Dannenberg,  1992;  Laurel,  1990;  Poison, 
1988,  Shneiderman,  1987). 

A  proposal  was  written  and  funding  for  the  portable  usability  lab  was  obtained  in  1993.  Usability 
Systems,  Inc.,  an  Atlanta  based  company  that  builds  usability  labs  for  testing  software,  responded  to  our 
request  for  bids  for  a  portable  system  with  the  "Luggage  Lab  2000"  (see  Figure  1).  This  lab  is 
transportable  to  any  site  where  interactive  multimedia  or  any  other  type  of  software  is  being  used  for 
education,  training,  information,  or  performance  support  purposes.  The  "lab'  includes  a  remote-controlled 
video  camera  that  can  be  focused  on  the  user's  face,  work  on  a  desk,  the  user's  computer,  keyboard  and 
mouse,  or  any  other  aspect  of  the  user  environment  considered  important  in  the  study.  The  system 
r'  nultaiieously  records  the  user's  computer  screen.  Researchers  sit  at  a  control  panel  that  allows  them  to 
ouserve  the  user(s)  directly  or  on  any  of  the  video  screens  displaying  selected  aspects  of  the  context. 
Researchers  can  control  what  is  recorded,  e.g.,  most  of  the  user's  screen  along  with  a  small  insert  image  of 
the  user's  facial  expressions  or  body  language. 
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Figure  1.  Portable  usability  lab.  (FMI,  contact  Usability  Systems,  Inc.,  404/475-3505.) 
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Commercial  software  developers  have  employed  "usability  labs"  for  formative  evaluation  of  software 
applications  for  many  years  (Gomoll,  1990).  For  example,  both  AT&T  and  IBM  maintain  fixed  usability 
testing  labs  in  the  Atlanta  area.  These  fixed  usability  labs  generally  consist  of  two  rooms  separated  by  a 
one-way  glass  window  (see  Figure  2  below).  In  one  room,  a  computer  user  sits  at  a  desk  and  uses  the 
application  being  evaluated,  e.g.,  a  new  spreadsheet  program.  Several  video  cameras  mounted  in  the  room 
are  focused  on  various  aspects  in  the  room.  In  the  other  room,  evaluators  and  designers  sit  at  control  panels 
where  they  can  simultaneously  observe  the  user  in  the  room  through  the  one-way  glass  or  any  of  the  video 
screens  displaying  selected  aspects.  The  user  may  be  instructed  to  "think  aloud"  as  he/she  uses  the  program, 
e.g.,  talk  about  why  certain  choices  are  made  or  describe  any  confusion  about  the  program's  interface. 
Alternatively,  the  evaluators  may  question  the  user  via  headsets  or  speakers  about  why  he/she  has  done 
certain  actions.  Typically,  these  sessions  are  videotaped  for  later  analysis  and  documentation.  Some  fixed 
usability  labs  feature  a  third  room  where  clients  can  observe  the  usability  testing  as  it  is  being  conducted. 


Figure  2.  Fixed  usability  laboratory  (artwork  by  Lih-Juan  Chanlin). 


The  portable  software  usability  lab  is  patterned  after  these  commercial  labels,  but  rather  than  forcing  users 
to  come  to  a  lab  and  test  software  in  an  artificial  environment,  the  portable  lab  allows  the  users  to  stay  in 
their  own  environment.  We  believe  that  this  increases  the  validity  of  both  software  testing  and  research 
studies.  This  paper  is  intended  to  describe  our  procedures  for  using  this  research  tool,  present  some  of  the 
preliminary  results  we  have  obtained  with  it,  and  suggest  recommendations  for  further  research. 


Procedures  for  Usability  Testing 

The  portable  usability  lab  enables  researchers  to  collect  both  quantitative  and  qualitative  data  related  to 
issues  such  as  user  interface,  mental  models,  navigation,  documentation  utility,  effectiveness,  and 
efficiency.  A  variety  of  research  and  evaluation  protocols  are  possible  using  this  type  of  tool  (Hix  & 
Hartson,  1993;  Nielsen,  1993). 

Our  usability  lab  comes  in  four  large  cases  that  stack  onto  a  dolly  for  rolling  (see  Figure  1 ).  Detailed 
instructions  describe  step  by  step  procedures  for  connecting  system  components  together.  All  cables  are 
color  coded  to  assist  in  setup.  Beyond  the  normal  components  of  the  Luggage  Lab  2000,  Usability 
Systems,  Inc.  added  picture  in  a  picture  capability  to  the  lab.  This  allows  us  to  record  both  the  full 
computer  screen  and  an  inset  of  the  user.  An  FM  microphone  is  provided  for  think  aloud  and  interview 
procedures.  The  camera  has  remote  control  features  to  pan  and  tilt  for  added  flexibility  in  recording  the 
interactions  of  the  user. 

This  flexible  system  can  be  used  to  support  and  enhance  many  methods  of  usability  testing.  Nielsen  ( i  993) 
identified  the  following  methods  for  gathering  usability  data:  observation,  think  aloud,  questionnaires, 
interviews,  focus  groups,  logging  actual  use  and  user  feedback.  Researchers  and  developers  should  select  the 
appropriate  methods  to  collect  data  to  address  different  usability  issues  and  questions. 
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Each  of  these  methods  has  different  strengths  and  weaknesses,  and  combining  different  methods  is  often 
necessary  to  improve  overall  usability  testing.  For  example,  you  might  want  to  ask  questions  to  users 
during  observations,  but  asking  questions  during  an  observation  can  change  what  the  user  would  naturally 
do.  One  solution  is  to  record  the  user  with  the  portable  usability  lab,  and  later  play  the  tape  back  to  the 
user  and  ask  questions.  The  tape  assists  the  user  in  recalling  the  recorded  session.  In  addition  the  same  tape 
can  be  shown  to  human  factors  experts  for  their  advice  and  interpretations.  A  focus  group  can  review 
videotapes  of  users  in  their  actual  working  conditions  to  stimulate  discussions. 

Both  research  projects  and  software  evaluations  can  involve  the  use  of  experts  to  judge  the  performance  of 
users  on  various  types  of  tasks.  Reliability  is  an  important  issue  whenever  human  judges  are  used.  Having 
videotaped  data  scored  by  multiple  experts  can  provide  reliability  information  about  the  data  collection 
process.  Collecting  data  about  benchmark  tasks  is  another  use  of  the  portable  usability  lab.  A  benchmark 
task  is  a  common  activity  the  user  performs  with  the  technology.  These  benchmarks  are  selected  by  the 
developer  to  measure  quantitatively  the  interface  design  (Hix  &  Hartson,  1993).  The  usability  system  can 
record  the  user's  performance  on  benchmarks  for  later  analysis. 

In  addition  to  these  planned  benchmark  tests,  you  may  observe  users  with  the  portable  usability  system 
over  time  in  their  natural  environment  doing  what  they  select  to  do.  This  is  more  like  naturalistic  research 
than  benchmark  evaluation.  Benchmarks  are  focused  and  efficient  whereas  natural  observation  is  time 
consuming  and  less  directed.  However,  the  generalizability  of  findings  from  naturalistic  research  may  be 
greater.  Both  are  valuable  and  can  be  supported  with  the  portable  usability  system. 

Many  times  in  designing  an  interface  you  may  have  multiple  options  for  designing  an  interaction  with  the 
computer.  Local  "rapid  prototyping"  is  the  creation  of  multiple  designs  for  mall  components  of  your 
software  (Hix  &  Hartson,  1993).  By  comparing  tasks  performed  with  each  option  of  interacting,  you  can 
feel  more  confident  about  which  design  to  use. 

An  example  of  multiple  designs  for  a  computer  interaction  was  considered  by  the  first  author  in  the  design 
of  an  angle  measurement  tool,  a  computerized  protractor.  One  way  to  measure  an  angle  is  to  click  on  three 
separate  points  that  form  an  angle  (one  on  the  first  leg,  one  at  the  vertex.,  and  one  on  the  other  leg  of  the 
angle).  Another  way  to  design  the  protractor  is  to  click  on  the  vertex  and  drag  along  one  side  and  release, 
then  you  push  or  pull  to  create  a  spanning  angle  that  reaches  the  other  leg  (Hale,  Gustafson  &  Yeany, 
1989).  The  first  interaction  is  more  straight  forward.  The  latter  has  the  advantage  of  positive  and  negative 
values  depending  visually  on  the  direction  of  spanning.  By  creating  both  versions  in  a  small  prototype, 
you  may  test  thesy  for  inclusion  into  the  final  product. 

Finding  appropriate  users  who  will  allow  you  to  videotape  them  at  work  with  a  prototype  system  is 
sometimes  difficult.  Experts  suggest  only  using  a  few  participants  that  match  your  target  users  (Hix  & 
Hartson,  1993).  You  should  supplement  these  by  using  one  or  two  experts  in  human-computer  interface 
design  to  review  your  software.  With  so  few  people  interacting  with  your  softwaie,  you  may  want  to 
inaximive  the  data  you  have  by  doing  detailed  analysis  using  a  tool  such  as  Videotape  Analyzer  (Tsao,  Hale 
&  Fan,  1 394).  This  software  tool  supports  repeated  detailed  qualitative  analysis  of  video.  The  portable 
usability  lab  is  ideal  for  producing  videotapes  that  can  be  analyzed  with  this  tool. 


Results  of  Usability  Testing 

The  usability  lab  at  UGA  was  used  in  late  1994  for  two  different  purposes,  i.e.,  formative  evaluation  and 
educational  research.  The  first  application  was  part  of  a  formative  evaluation  of  a  computer-based  learning 
environment  for  statistics  called  StatSim.  The  second  was  to  examine  the  types  of  interactions  initiated  by 
learners  who  were  in  two  different  experimental  groups  using  an  intelligent  tutoring  system  called  the 
MSRT  Tutor.  The  best  way  to  understand  the  power  of  the  usability  lab  is  to  describe  our  experiences  with 
these  two  applications. 

In  th".  case  of  the  StatSim  evaluation,  the  usability  lab  was  used  in  the  College  of  Education  at  UGA. 
Therefore,  the  portability  part  of  the  system  was  not  utilized  to  its  fullest  extent.  The  system  was  setup  in 
a  small  limited  access  hallway  that  was  adjacent  to  the  room  where  the  software  testing  was  to  take  place. 
This  was  quite  useful  in  that  the  people  who  were  being  recorc'  -d  did  not  actually  see  any  of  the  equipment 
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that  was  being  used.  We  believe  that  this  would  have  resulted  in  a  heightened  anxiety  level  within  the 
people  who  were  learning  from  StatSim.  However,  because  there  was  a  cinder  block  wall  between  the 
usability  equipment  and  the  learner,  the  audio  transmit/receive  ability  was  problematic.  Later  uses  of  the 
lab  attended  to  this  placement  problem  by  making  sure  that  there  were  no  obstructions  between  the  remote 
microphone  of  the  user  and  the  receiver  built  into  the  usability  equipment.  Office  partitions  block  the 
view,  but  do  not  block  the  radio  signal.  Another,  as  yet  unexplored  possibility,  would  be  to  add  to  the  lab 
a  remote  antenna  that  can  be  placed  in  the  room  where  the  user  is  located. 

The  intent  of  the  StatSim  evaluation  was  to  improve  the  software.  The  program  had  been  developed  by  Liu 
Zhang,  a  graduate  student  in  the  Artificial  Intelligence  Program  at  UGA.  This  program  was  part  of  his 
requirements  for  a  Master's  degree.  Within  the  program,  he  used  elements  of  constructivist  learning  theory 
to  allow  the  learner  to  freely  explore  problems  in  statistics.  However,  once  the  learner  navigated  to  the 
point  where  problem  solving  was  taking  place,  the  system  was  then  able  to  model  the  learner's  problem 
solving  and  provide  appropriate  feedback. 

The  StatSim  program  was  written  in  Visual  Basic.  Many  of  the  changes  that  were  made  to  the  software 
were  the  result  of  learners'  expressed  lack  of  understanding  of  particular  icons,  text  messages,  or  information 
windows.  While  it  would  be  beyond  the  requirements  of  this  paper  or  interest  of  the  reader  to  document  all 
the  changes  here,  it  should  be  noted  that  confusing  items  that  were  presented  to  the  learner  as  they  interacted 
with  the  StatSim  program  were  located  not  only  by  their  verbal  comments,  but  also  by  their  facial 
expressions  as  they  were  being  shown  different  aspects  of  the  program.  The  videos  captured  by  the 
usability  lab  enabled  us  to  locate  and  modify  these  confusing  items. 

Curiously,  during  the  interactions,  StatSim  caused  the  computer  to  "lock  up"  on  a  seemingly  unpredictable 
basis.  Each  time  this  would  happen,  the  computer  was  restarted  and  (he  learner  continued.  At  first,  the 
developer  was  at  a  loss  as  to  why  the  system  was  crashing.  Upon  review  of  the  video  tape  that  was  created 
by  the  usability  lab,  it  was  discovered  that  the  system  would  crash  on  the  fifth  time  that  the  learner  would 
access  the  calculator  (a  resource  program  that  comes  with  Microsoft  Windows).  Apparently,  each  time  the 
learner  accessed  the  calculator,  when  they  were  done,  they  would  click  back  on  a  StatSim  screen.  The 
calculator  program  was  therefore  still  running.  The  next  time  the  learner  clicked  on  the  calculator,  a  new 
instance  of  the  calculator  was  created.  Finally,  on  the  fifth  access  to  the  calculator,  the  memory  was 
exceeded  and  the  computer  would  "lock  up".  The  video  obtained  with  the  usability  lab  enabled  the  developer 
to  track  down  this  peculiar  problem. 

A  different  application  of  the  usability  lab  took  place  when  we  used  it  during  a.,  experiment  with  the 
MSRT  Tutor.  The  tutor  was  originally  developed  by  Michael  Orey  to  determine  if  an  interesting  ITS  like 
system  could  be  built  to  run  on  a  286-based  MS-DOS  computer  system.  It  has  since  become  a  primary 
tool  for  teaching  soldiers  how  to  operate  the  MSRT  radio.  It  has  been  used  by  over  10,000  soldiers  for  this 
purpose.  Some  of  these  soldiers  use  the  program  as  part  of  their  fo  mal  classroom  training,  others  use  the 
system  in  the  field  as  is  needed.  The  experiment  examined  various  sustainment  schedules  in  order  to  help 
soldiers  maintain  procedural  knowledge  in  memory.  One  group  used  the  Tutor  every  two  weeks,  another 
group  every  three  weeks,  and  the  last  group  every  six  weeks.  Not  surprisingly,  the  two  week  group  was 
better  than  the  three  week  group  that  was  better  than  the  six  week  group.  The  question  that  confronted  us 
was:  What  kinds  of  things  were  learners  doing  on  the  Tutor  in  each  of  these  three  groups?  We  used  the 
usability  lab  to  document  the  interactions  that  the  learners  made  on  the  system.  We  thought  that  'he 
differences  observed  would  help  us  understand  or  explain  the  learning  differences  that  were  measureo  between 
the  groups.  Two  learners  from  each  group  were  randomly  selected  to  use  the  computer  that  was  hooked  up 
to  the  usability  lab.  A  detailed  analysis  was  performed  on  each  of  the  video  tapes  that  were  created  while 
these  learners  interacted  with  the  Tutor. 

The  poriability  of  the  usability  lab  was  very  helpful  in  this  application.  This  experiment  took  place  at  Fort 
Gordon,  Georgia,  a  ninety-minute  drive  from  UGA.  The  usability  lab  was  loaded  into  a  small  car  (Ford 
Escort)  driven  by  the  second  author  of  this  paper.  The  entire  system  breaks  down  into  four  large  suitcases. 
One  was  placed  in  the  trunk,  two  in  the  back  seat  and  one  in  the  front  passenger  seat.  A  classroom  at  Fort 
Gordon  was  used  for  data  collection.  The  computer  was  setup  in  the  front  of  the  room  facing  forward  and 
the  usability  lab  was  behind  the  learner.  We  did  not  want  to  make  the  learner  feel  too  much  anxiety  while 
we  collected  the  data,  but  this  setting  required  us  to  place  the  lab  in  the  same  room  as  the  learner.  On  the 
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other  hand,  this  configuration  did  not  have  the  same  transmit/receive  problems  that  we  had  experienced  with 
the  formative  evaluation  in  the  College  of  Education 

The  results  of  the  Tutor  analysis  resulted  in  two  important  findings.  First,  the  six  and  three  week  groups 
performed  many  more  interaction  patterns,  an  indication  that  the  learners  were  forgetting  how  to  use  the  program 
(the  six  week  people  more  so  than  the  three  week  people).  This  type  of  interaction  would  not  result  in  better 
performance  on  the  actual  task.  In  fact,  knowing  how  to  use  the  Tutor  per  se  is  of  little  benefit  to  performing 
the  task. 

The  second  type  of  interaction  that  was  different  between  groups  requires  a  little  more  explanation  of  the 
treatments.  The  two  week  group  performed  the  tasks  twice  every  two  weeks  for  six  weeks  (a  total  of  2  X  3 
or  six  times).  The  three  week  group  performed  the  tasks  three  times  every  three  weeks  for  six  weeks  (a 
total  of  3  X  2  or  six.  times).  The  six  week  group  performed  the  task  six  times  once  on  the  sixth  week 
following  the  initial  training.  In  this  way,  we  could  vary  the  schedule  while  holding  the  number  of 
repetitions  constant.  The  result  of  this  design  as  shown  in  the  videos  produced  by  the  usability  lab  was  that 
there  were  diminishing  benefits  on  the  repetitions  as  the  number  increased.  The  sixth  time  the  six  week 
group  performed  the  task  was  largely  a  non  memory  intensive  effort  on  the  part  of  the  learner  (what  youth 
today  call  a  "no  brainer").  According  to  this  data,  one  might  conclude  that  if  we  were  to  separate  the 
repetitions  by  a  short  break  for  the  six  week  group,  their  effort  at  the  task  might  improve  and  so  would 
their  performance. 

This  study  also  revealed  statistically  significant  differences  in  outcomes  between  soldiers  who  performed  a 
fixed  number  of  problems  and  those  that  worked  towards  a  criterion  level  of  performance  during  each  of  their 
three  sustainment  episodes.  It  was  difficult  to  understand  why  we  would  g^t  statistical  differences  between 
the  groups  until  we  went  back  and  examined  the  data  on  the  video  tapes  to  compare  learners  in  each  group. 
As  it  turns  out,  we  only  had  data  on  two  of  subjects  from  the  Fixed  condition  and  only  one  subject  from 
the  Criterion  condition.  The  results  of  our  analysis  are  presented  in  Table  1 .  There  were  three  primary 
issues  or  categories  of  behaviors  that  were  observed  during  their  interaction.  These  issues  clearly 
demonstrate  that  the  Criterion  One  person  is  focusing  his  efforts  on  meeting  the  criteria  imposed  by  his 
version  of  the  tutor.  If  this  pattern  can  be  generalized  to  the  entire  group,  then  a  possible  explanation  for 
why  there  was  a  performance  difference  is  that  the  criterion  group  appears  to  be  focusing  their  attention  and 
efforts  towards  meeting  the  criteria  while  the  Fixed  group  is  focusing  their  attention  and  effort  more  directly 
on  learning  the  content  of  the  Tutor.  If  this  assumption  is  true,  it  is  clear  why  we  found  this  performance 
difference  and  it  is  clear  that  having  learners  perform  a  fixed  number  of  trials  is  a  preferred  teaching  strategy 
for  sustainment  learning. 

Table  1. 

Comparson  between  two  subjects  in  the  Fixed  group  and  one  subject  in  the  Criterion  group 


Issue  Fixed  One  Fixed  Two  Criterion  One 


View  of  Coach 

Ask  if  Click  Coach  OK; 
Uses  Coach  frequently; 
Says  he,  "needs  Coach" 

Says  he  already  knows 
about  MSRT;  Feels  he 
doesn't  need  Coach; 
Only  uses  Coach  once 

Avoids  Coach; 
Recognizes  that  Coach 
will  prevent  progress 

Learning  Goals 

Finishes  the  2  passes 
and  expresses  relief; 
Doesn't  want  to  do  it 
again 

Tries  to  finish  within  5 
minutes 

Is  totally  focused  on 
meeting  the  criterion  for 
completing  the  Tutor 

Expressions  of 
Confidence 

Creates  many  excuses 
for  errors 

"I'm  surprised  I 
remember" 

Anticipating 
performance  feedback, 
"Do  I  have  any  errors?" 

While  it  is  difficult  to  fully  capture  the  entirety  of  the  unpad  of  the  usability  lab  in  these  short  anecdotes, 
we  hope  that  the  reader  begins  to  understand  the  potential  benefit  of  this  type  of  research  tool.  We  have 
several  new  research  projects  underway  where  the  usability  lab  will  have  great  benefit.  For  example,  one  of 
our  doctoral  students  is  focusing  list  research  on  the  mental  models  users  construct  of  an  on-line  public 
access  catalog  (OPAC)  (Kelly.  1994).  She  is  using  the  portable  usability  lab  to  record  users  as  they 
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perform  searches  on  GALIN  (Georgia  Library  Information  Network)  in  the  main  library  where  she  works  as 
a  reference  librarian.  She  then  reviews  each  video  with  a  user,  asking  them  to  explain  why  they  made 
certain  choices.  Research  on  mental  models  is  an  important  direction  in  the  study  of  user  interfaces  for 
interactive  multimedia  and  other  applications  of  instructional  technology  (Jih  &  Reeves,  1992;  Leiser, 
1992). 


Conclusion 

The  importance  of  the  "usability"  approach  to  research  and  evaluation  is  considerable  (Nielsen,  1993). 
Currently,  instructional  designers  and  researchers  have  an  inadequate  base  of  knowledge  about  how  users 
react  to  and  learn  with  multimedia  programs  and  other  types  of  computer  programs  such  as  electronic 
performance  support  systems  (Gery,  1991).  The  research  data  yielded  by  the  portable  usability  lab  provides 
an  improved  basis  for  guiding  the  design  and  implementation  of  instructional  technology  systems.  For 
example,  we  believe  that  the  enhancement  of  our  understanding  of  interactive  multimedia  user  interfaces  can 
improve  the  dissemination,  implementation,  and  effects  of  using  multimedia  at  all  levels  of  education  and 
training.  Initial  marketing  of  multimedia  has  succeeded  primarily  on  the  basis  of  selling  the  "bells  and 
whistles"  of  the  technology,  but  now  school  boards,  superintendents,  parents,  and  taxpayers  are  beginning 
to  demand  research-based  evidence  that  multimedia  enhances  learning.  Fundamental  understanding  of  how 
interactive  multimedia  is  used  by  teachers,  students,  trainees,  etc.  is  an  essential  part  of  that  evidence.  The 
portable  usability  testing  lab  provides  us  with  precisely  that  kind  of  evidence. 
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Introduction 


Hypermedia  is  a  relatively  new  technology,  and  data  on  the  most  effective  ways  to  use  hypermedia 
in  training,  while  steadily  growing,  are  still  relatively  scattered.  Hypermedia,  because  of  its  greatly 
expanded  capabilities  over  traditional  computer-based  training,  presents  new  challenges  for  designers, 
developers,  and  evaluators  of  training  as  well  as  trainers.  Issues  arise  in  the  design,  development,  and 
delivery  of  hypermedia  training  which  were  not  present  with  traditional  computer-based  training  and 
therefore  were  not  given  extensive  treatment  either  in  the  development  of  theoretical  models  of  computer- 
based  instructional  design  or  in  the  design  of  such  instructional  materials.  Examples  df  the  additional 
complexities  which  must  be  addressed  in  designing  and  delivering  effective  hypermedia-based  training 
include:  the  appropriate  type  and  degree  of  structure  to  be  incorporated  into  the  information  content  of 
hypermedia  applications;  the  extent  and  type  of  navigation  options  to  be  provided  to  the  user;  the 
appropriate  role  of  the  trainer  or  facilitator;  and  the  appropriate  mix  of  different  media  (text,  computer 
graphics,  computer-generated  animation,  still  video  graphics  and  photographs,  motion  video,  digitized 
speech,  other  audio,  etc.). 

Use  of  Qualitative  Research  Methodologies 

Because  hypermedia  training  is  so  new  and  so  complex,  it  is  difficult  to  identify  appropriate 
research  hypotheses  which  can  be  investigated  using  quasi-experimental  techniques.  Furthermore,  the 
complexity  of  hypermedia  training  used  in  trainer-facilitated  group  settings  (the  type  of  hypermedia  training 
investigated  in  this  study)  makes  it  difficult  if  not  impossible  to  control  adequately  for  plausible  rival 
hypotheses.  For  these  reasons,  these  researchers  feel  that  qualitative  methodologies  constitute  a  more 
appropriate  approach  for  the  investigation  of  the  effectiveness  of  hypermedia  training. 

Qualitative  approaches  provide  a  broader  perspective  on  actual  applications  of  hypermedia  training 
in  real-world  settings.  This  approach  was  chosen  because  the  researchers  recognized  that,  at  this  stage  in  the 
development  of  hypermedia  technology,  we  do  not  have  a  sufficiently  well-developed  understanding  of 
which  variables  related  to  the  use  of  hypermedia  are  most  appropriate  to  investigate,  and  how  to 
systematically  control  these  variables  in  quasi-experimental  studies.  Use  of  qualitative  methodologies  also 
represents  an  acknowledgment  that  the  researcher  can  not,  indeed  may  not  want  to,  exert  complete  control 
over  the  training  situation.  In  fact,  attempts  to  impose  undue  control  over  the  delivery  of  training  may 
introduce  distortions  in  the  way  hypermedia  is  used  which  will  negatively  effect  the  validity  of  the  data 
collected.  At  this  point  in  the  development  of  our  understanding  of  how  to  use  hypermedia  effectively  for 
training,  it  seems  that  the  most  important  question  to  be  addressed  is,  "What  is  going  on  when  hypermedia 
is  used  for  training?" 

Use  of  Computer  Tracking  Tools 

On  the  other  hand,  quantitative  data  on  the  uses  of  hypermedia  for  training,  when  they  can  be 
collected  unobtrusively  and  without  artificially  distorting  the  delivery  of  training,  can  provide  the  foundation 
for  analysis  which  supplements  the  qualitative  observations  and  contributes  to  their  validity,  or  which  flags 
results  which  may  appear  to  be  conflicting  and  which  require  further  investigation.  Of  particular  value  is 
the  collection  uf  quantitative  data  on  the  extent  and  specific  ways  in  which  trainees  use  hypermedia  in  a 
specific  training  program.  This  can  be  accomplished  through  the  use  of  computer-based  tracking  tools 
incorporated  into  the  hypermedia  application  used. 

Computer  tracking  tools  can  record  the  number  and  type  of  inputs  (keyboard  entry,  mouse  pointing 
or  clicking,  etc.)  into  the  hypermedia  program  by  the  user  at  any  point  in  the  program,  as  well  sr  the  exact 
time  and  duration  of  the  interaction.  This  information,  then,  provides  a  comprehensive  recor  J  of  how  a 
specific  user  interacted  with  the  hypermedia,  including  the  paths  taken  by  the  user  from  screen  to  screen,  and 
the  type,  time,  and  duration  of  the  interactions. 

Computer  tracking  tools  have  been  used  by  a  number  of  researchers  for  formative  evaluation  and 
for  forma!  research  on  attributes  of  hypermedia  training.  Gay  and  Mazur  (1993)  described  several  different 
types  of  computer  tracking  tools,  ranging  from  tools  which  produce  primarily  text-based  lists  to  tools 
which  produce  graphical  and/or  pictorial  depictions  of  users'  actions.  They  described  how  they  used  these 
tools  in  the  investigation  of  hypermedia  programs  developed  at  Cornell's  Interactive  Multimedia  Group. 
Misanchuk  and  Schwier  ( 1991 )  identified  three  distinct  uses  for  computer  tracking  tools,  the  outcomes  of 
which  they  referred  to  as  audit  trails.  These  use;;  included:  1 )  data  collection  for  formative  evaluation;  2) 
tools  for  basic  research  into  instructional  design  of  multimedia;  3)  auditing  use  of  multimedia  in  a  public 
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forum.  They  describe  several  quantitative  and  qualitative  methods  for  describing  and  analyzing  computer- 
generated  audit  trails. 

The  developers  of  the  Cities  in  Schools  (CIS)  hypermedia  training  curriculum  recognized  the 
potential  of  incorporating  computer  tracking  tools  into  the  CIS  hypermedia  software  early  in  the  project. 
Design  specifications  of  the  hypermedia  included  an  analysis  of  the  types  of  data  to  be  collected  and  the 
mechanisms  for  collecting  that  data  automatically  (Story  and  Harvey,  1990). 

Use  of  Hybrid  Methodologies 

This  paper  will  describe  a  research  effort  which  investigated  the  effectiveness  of  the  CIS 
hypermedia  training  course  for  senior  executives  of  that  national  dropout  prevention  program.  The  study 
employed  a  hybrid  methodology  of  qualitative  techniques  and  computer  tracking  tools,  in  order  to  provide  as 
complete  a  picture  as  possible  of  how  the  hypermedia  training  course  was  delivered  and  received. 


The  Cities  in  Schools  Hypermedia  Training  Curriculum 

The  authors  have  been  involved,  for  the  past  six  years,  in  the  design,  development,  and 
implementation  of  an  integrated  hypermedia  training  curriculum  for  Cities  in  Schools,  Inc.  The  CIS 
training  curriculum  was  developed  by  faculty  and  staff  of  the  Advanced  Information  Technologies 
Laboratory  (AITL)  in  Lehigh  University's  College  of  Education. 

The  CIS  integrated  hypermedia  training  curriculum  was  developed  by  a  team  which  included 
instructional  design  specialists;  designers  and  producers  of  video,  computer,  and  print  training  materials;  and 
experts  in  adult  learning.  Three  courses,  comprising  a  total  of  ten  weeks  of  full-day  training,  have  been 
developed.  The  CIS  training  curriculum  :>  now  being  delivered  by  CIS  trainers  (trained  by  AITL  staff)  at 
the  CIS  National  Training  Institute  at  Lehigh  University  and  at  six  regional  CIS  training  centers  around  the 
country. 

The  Cities  in  Schools  hypermedia  training  curriculum  uses  a  training  paradigm  different  from  the 
one-person/one  computer  paradigm  of  traditional  computer-based  training.  Instead,  a  novel  approach  to  the 
use  of  hypermedia  for  training  is  employed  in  which  the  hypermedia  materials  are  used  as  training  resources 
to  facilitate  interpersonal  interactions  in  trainer- facilitated  group  learning  activities. 

The  OS  training  course  investigated  for  this  study  was  the  Program  Operations  course,  designed  to 
train  senior  management  personnel  who  are  executive  directors  of  city-wide  or  county-wide  CIS  programs. 
The  course  included  a  comprehensive  (2500-screen)  hypermedia  software  program,  a  one-hour  videodisk,  and 
trainers'  and  trainees'  printed  materials.  A  simulated  city  with  a  CIS  dropout  prevention  program  were 
incorporated  into  the  training  materials.  A  trainer's  resource  book  included  objectives,  learning  strategies, 
and  evaluation  strategies  for  32  learning  activities  which  incorporated  a  total  of  1 17  separate  learning  sub- 
activities. 

The  CIS  Program  Operations  course  supported  multiple  modes  of  learning,  including: 

•  presentations  by  the  trainer; 

•  examination  of  computer,  print,  and/or  video  materials  by  individuals,  pairs,  or  small  groups  of 
trainees; 

•  small  group  and  large  group  discussions; 

•  role  playing; 

•  other  group  activities,  and 

•  individualized,  self-paced  learning  activities. 

Design,  development  and  delivery  of  CIS  hypermedia  training  was  guided  by  the  Advanced 
Information  Technologies  Laboratory  Process  Model  (Harvey,  1993).  According  to  this  model,  learning 
proceeds  from  lower  level  activities  (information  acquisition  and  comprehension)  to  higher  level  (analysis 
and  synthesis;  application  in  simulated  settings),  and  culminates  in  a  systematic  application  of  the 
knowledge  to  a  real-world  situation. 


Purpose  of  the  Study 

The  authors  recognized  that,  since  hypermedia  is  a  new  tool  for  training,  a  comprehensive  body  of 
knowledge  describing  how  hypermedia  can  best  be  used  for  training  has  not  yet  been  developed.  In  fact,  the 
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salient  characteristics  of  an  appropriate  training  program  which  utilizes  hypermedia  training  have  not  yet 
been  identified.  The  Cities  in  Schools  hypermedia  training  course  represented  the  collective  knowledge  and 
wisdom  of  the  AITL  development  group.  The  purpose  of  the  research  study  reported  in  this  paper  was  to 
validate  the  approach  taken  in  developing  CIS  hypermedia  training,  and  to  identify  aspects  of  using 
hypermedia  for  training  which  would  warrant  closer  examination.  Specific  goals  of  the  qualitative 
component  of  this  research  study  are  presented  in  the  following  sections. 

How  is  Hypermedia  Used  in  a  "Real-World"  Setting? 

Since  hypermedia  is  a  new  technology,  and  the  approach  taken  by  the  CIS  hypermedia  curriculum 
was  novel,  researchers  wanted  to  determine  and  describe  in  detail  how  the  hypermedia  curriculum  was  used 
in  an  actual  course  delivered  in  a  "real-world"  setting. 

What  Characteristics  of  Hypermedia  Warrant  Further  Research? 

The  puipose  of  the  study  was  to  describe,  as  much  as  possible,  every  aspect  of  the  week-long 
hypermedia  training  program.  Analysis  of  the  data  collected  would  serve  as  a  foundation  for  identifying 
specific  questions  and  hypotheses  about  teaching  and  learning  with  hypermedia  which  could  be  investigated 
in  future  research. 

Specific  goals  of  the  computer  tracking  tools  component  of  this  research  study  are  described  in  the 
following  sections. 

How  Frequently  and  to  What  Extent  Did  Trainers  and  Trainees  Utilize  Hypermedia 
Materials? 

The  CIS  Program  Operations  training  course  was  designed  as  an  integrated  hypermedia  training 
course.  The  hypermedia  materials  were  designed  to  present  information,  provide  guidance  and  support 
materials  for  comprehension  and  higher-level  (analysis/synthesis  and  application  activities)  learning 
activities.  The  materials  provided  resources  and  tools  for  both  trainer  and  trainee  in  the  knowledge  and 
skills  required  to  function  effectively  as  an  executive  director  of  a  city-wide  or  county-wide  Cities  in 
Schools  program.  Trainer-facilitated  learning  activities  included  role-playing  and  small-group  and  large- 
group  discussion  used  in  conjunction  with  and  as  follow-up  to  interaction  with  the  hypermedia  materials. 
The  researchers  were  interested  in  the  frequency  and  amount  of  time  specific  groups  of  trainees  spent 
actually  interacting  with  the  hypermedia  materials  compared  to  the  amount  of  time  spent  in  other  activities 
away  from  the  hypermedia  workstations. 

Which  Paths  Did  Different  Users  Employ  in  Exploring  the  Hypermedia  Materials? 

Structures  of  hypermedia  applications  vary  widely,  from  relatively  sequential  to  totally 
nonsequential.  The  CIS  hypermedia  application  investigated  employed  a  hierarchical  structure,  in  which 
computer,  print,  or  video  learning  resources  were  grouped  into  learning  sub-activities  which  addressed  a 
specific  learning  objective.  Ordinarily,  trainers  directed  trainees  to  examine  some  or  all  learning  resources 
for  a  specific  learning  sub-activity.  However,  resources  for  all  of  the  learning  activities  were  available  at 
any  time,  and  users  could  access  them  by  selecting  the  main  screen  for  any  learning  activity.  In  addition, 
user  tools  in  the  CIS  hypermedia  application  included  a  "personal  hypermedia  list"  option,  w-hich  provided 
the  capability  to  navigate  directly  to  a  computer  resource  which  had  been  added  to  the  user's  personal  list. 
Thus,  each  trainee's  "path"  through  the  hypermedia  was  potentially  different.  One  purpose  of  this  study 
was  to  examine  the  actual  trainee  use  of  the  hypermedia  in  a  week-long  training  program. 

What  Relationships  Were  Present  Among  the  Ways  Hypermedia  Materials  Were  Used, 
the  Extent  and  Type  of  Non-Hypermedia  Activities,  and  the  Attainment  of  Specific 
Learning  Objective    of  the  Hypermedia  Training  Course? 

Specific  learning  objectives  are  included  for  each  of  the  1 17  learning  sub-activities  included  in  the 
CIS  Program  Operations  training  course.  For  each  learning  objective,  matching  evaluation  strategies  are 
presented  which  specify  criteria  which  would  indicate  that  the  learning  objective  had  been  met,  and  where  to 
look  for  such  evidence  (for  example,  in  trainees'  comments  during  small-  or  large-group  discussions, 
written  statements  by  students  made  on  activity  sheets,  or  the  trainees'  written  summary  action  plans). 
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Learning  objectives  and  learning  strategies  are  clearly  specified  in  the  resource  book  provided  to  each  trainer. 
Since  the  overall  goal  of  CIS  Program  Operations  training  is  to  help  trainees  develop  the  sum  of  knowledge 
and  skills  which  the  learning  objectives  represent,  it  was  important  to  demonstrate  that  this  goal  had  been 
achieved.  In  addition,  the  researchers  were  interested  in  whether  any  relationships  emerged  between  the  ways 
hypermedia  was  used  in  training  and  the  degree  to  which  training  objectives  were  met.  For  example,  would 
a  higher  proportion  of  trainees  meet  learning  objectives  when  trainers  made  extensive  use  of  the  hypermedia 
than  when  trainers  concentrated  on  other,  non-hypermedia  training  activities? 


Method 

A  hybrid  methodology  was  employed  in  this  study.  The  qualitative  methodology  employed  was 
developed  during  two  pilot  tests,  then  refined  for  use  in  the  study  of  a  week-long  training  course.  All 
aspects  of  this  use  of  hypermedia  were  examined,  including  the  demographics  of  the  participants,  trainees' 
and  trainers'  use  of  the  hypermedia  materials,  their  use  of  supplemental,  trainer-developed  materials,  and  the 
interactions  among  participants  (on-task  and  off-task,  during  training  sessions,  and  during  social  activities). 

Subjects 

Subjects  for  the  study  were  the  thirteen  trainees  and  three  professional  trainers  who  participated  in  a 
week-long  training  course.  The  trainees,  all  of  whom  had  volunteered  for  the  course,  varied  widely  in  age, 
race,  level  of  education,  and  amount  of  experience  or  knowledge  of  CIS  operations.  None  of  the  trainees 
expressed  noticeable  computer  anxiety. 

Trainers  for  the  course  were  Cities  in  Schools  trainers.  All  three  had  extensive  professional 
training  experience,  either  with  Cities  in  Schools  or  with  other  organizations.  All  had  taught  the  course  at 
least  once  previously,  although  their  depth  of  experience  with  this  particular  course  varied. 

Qualitative  Methodologies 

The  observer  (Nelson)  used  non-participant  observation,  a  method  which  is  applied  by  many 
researchers  in  the  social  sciences  (Delamont,  1992;  Miles  &  Huberman,  1984).  The  observer  took 
extensive  hand-written  notes  while  observing  training.  Observations  were  recorded  in  brief  descriptions  and 
were  subsequently  expanded  upon.  The  handwritten  notes  were  supplemented  by  audiotape  recording.  The 
notes  and  tapes  were  reviewed  each  evening,  in  order  to  identify  patterns  and  unusual  events  which  should  be 
examined  more  closely  on  following  days. 

The  CIS  Hypermedia  Computer  Tracking  Tools 

Hypermedia  tracking  tools  were  designed  into  the  CIS  hypermedia  applications,  and  implemented 
at  the  same  time  the  application  itself  was  developed.  The  tools,  which  were  developed  as  SuperCard 
procedures,  continually  monitored  the  status  of  the  hypermedia  workstation.  Interactions  of  any  kind  (key 
press,  mouse  click,  viewing  video,  etc.)  were  recorded.  Time  of  day  was  also  continually  monitored,  and 
the  time  and  duration  of  all  activities  was  recorded. 

Data  from  the  computer  tracking  tools  was  automatically  collected  in  a  text  file.  Information 
collected  for  each  trainee  (or  pair  of  trainees  working  together  on  the  same  hypermedia  workstation) 
included: 

•  the  time  the  computer  was  first  turned  on; 

•  the  start  time  of  each  interaction  session; 

•  the  end  time  of  each  interaction  session; 

•  complete  information  on  any  user  input  (keypresses,  mouse-clicks,  etc.),  and  the  time  of  day  of  such 
interaction; 

•  the  start  time  of  each  period  when  the  computer  was  not  being  used; 

•  the  end  time  of  each  idle  period; 

•  summary  data  for  each  day's  hypermedia  computer  use. 

Data  from  the  raw  text  files  was  transported  into  a  spreadsheet,  and  cross-tabulations  of  total  usage 
and  tables  of  usage  patterns  were  generated  for  each  computer  workstation.  These  data  were  then  analyzed  in 
order  to  identify  possible  patterns  of  use. 


Results 


Resnlts  of  the  Qualitative  Component  of  the  Study 

Documentation  of  training  events  in  the  form  of  extensive  logs  and  other  information  collected 
during  the  course  of  the  five-day  training  course  provided  a  wealth  of  data.  Analysis  of  these  data  indicated  a 
number  of  topics  which  will  warrant  closer  examination  in  future  studies.  These  areas  for  further 
investigation  were  described  in  detail  elsewhere  (Nelson,  1994;  Nelson  and  Harvey,  1994),  and  will  be 
summarized  here. 

Diversity  of  the  trainees  and  grouping  issues.  The  wide  variation  in  background  and 
experience  suggests  that  grouping  trainees  of  similar  backgrounds  and  experience  levels  should  be 
investigated. 

Cooperative  learning.  Trainees  working  in  pairs  appeared  to  exhibit  more  frequent  and  higher 
quality  interpersonal  interactions.  Requiring  grouping  may  promote  cooperative  learning  and  increase 
training  effectiveness. 

Presentation  of  objectives.  Trainers  were  inconsistent  in  overtly  presenting  objectives  of 
learning  activities.  Further  study  is  needed  on  whether  consistently  and  clearly  presenting  objectives  will 
increase  the  probability  of  the  attainment  of  those  objectives. 

Formal  evaluation  of  learning.  Trainers  varied  widely  in  the  degree  to  which  they 
systematically  applied  recommended  evaluation  strategies.  Since  many  of  the  evaluation  strategies  focused 
on  comments  during  group  discussions,  it  was  difficult  to  assess  individual  attainment  of  objectives. 
Methods  are  needed  which  will  provide  evidence  of  the  degree  to  which  each  trainee  meets  training 
objectives,  while  at  the  same  time  not  hindering  the  flow  of  interpersonal  interactions  and  sharing  of 
learning. 

Applying  learning  to  real-world  situations.  Activities  designed  to  help  trainees 
systematically  apply  learning  from  training  to  their  own  job  situations  (by  developing  a  comprehensive 
"action  plan")  were  implemented  sporadically  by  trainers.  Further  study  of  trainees  after  they  have  returned 
to  their  own  job  situation  would  indicate  whether  the  results  of  training  are  having  a  significant  long-term 
effect  on  trainees'  behavior. 

Use  of  supplemental  trainer-developed  materials.  On  several  occasions,  trainers 
implemented  training  activities  of  their  own  design  in  lieu  of  the  activities  included  in  the  hypermedia 
curriculum.  The  content  and  quality  of  these  materials  varied  widely.  It  remains  to  be  seen  whether  the 
Hawthorne  effect  potential  of  trainer's  using  materials  they  developed  themselves  will  overcome  the  uneven 
quality  of  those  materials. 

Use  of  print  materials.  Although  the  CIS  hypermedia  course  attempted  to  integrate  the  use 
of  computer,  video,  and  print  materials  (e.g.,  by  including  a  "Print  resource"  icon  on  each  computer 
resource  screen  which  dictated  specific  pages  of  print  manuals),  trainers  only  occasionally  directed  trainees  to 
these  materials.  A  study  in  which  the  use  of  print  materials  is  carefully  controlled  and  monitored  would 
validate  the  usefulness  of  print  materials  in  the  context  of  hypermedia  training. 

Variations  in  trainers'  styles.  Trainers  varied  in  the  degree  of  apparent  comfort  with  and 
agreement  with  the  principles  of  hypermedia  training  incorporated  into  the  training  program.  A  study 
which  would  pre-assess  trainers'  preferred  instructional  styles,  monitor  the  styles  actually  employed,  and 
measure  the  degree  of  trainees'  attainment  of  objectives,  could  identify  the  trainer  styles  which  were  most 
effective  in  specific  situations  and,  more  importantly,  those  trainers'  styles  which  may  impede  trainees' 
learning.  Such  a  study  would  of  necessity  include  a  larger  number  of  trainers,  and  may  be  difficult  to 
implement,  since  control  of  variations  among  trainee  groups  would  be  problematic. 

Time  management.  Trainers  in  this  particular  course  appeared  to  have  difficulties  managing 
training  time.  Estimates  of  time  needed  for  specific  learning  activities  often  proved  inaccurate.  Logistics, 
(e.g.,  limiting  time  spent  on  breaks  to  the  specified  period)  also  impacted  the  training  schedule.  In  this 
case,  it  appears  that  additional  trainer  training  is  warranted,  rather  than  a  formal  investigation,  since  the 
deleterious  effects  of  poorly  managed  training  time  are  obvious. 

Course  evaluation.  Trainers  evaluated  trainees'  perceptions  of  the  hypermedia  training  course 
by  daily  "process  check"  open  discussions  and  by  a  Likert-type  reaction  form  on  the  final  day  of  training. 
Each  of  these  methods  have  significant  shortcomings.  More  valid  and  more  reliable  course  evaluation 
methods  need  to  be  developed  and  tested  in  future  studies  of  hypermedia  training. 
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Results  of  the  Computer  Tracking  Component  of  the  Study 

The  computer  tracking  component  of  this  study  was  more  exploratory  in  nature  than  the  qualitative 
component.  The  computer  tracking  tools  produced  a  voluminous  amount  of  raw  data.  Because  of  the  vast 
amount  of  data,  data  reduction  and  subsequent  analysis  have  proceeded  at  a  slower  pace  than  originally 
expected.  Preliminary  results  of  those  parts  of  data  analysis  which  have  been  completed  to  date  are 
presented  in  the  following  section.  The  completed  results,  when  they  have  been  compiled,  will  be  reported 
in  a  future  paper.  Data  are  simply  reported  here.  No  conclusions  are  drawn  from  them. 

Frequency  and  extent  of  use  of  hypermedia  materials.  Table  1  presents  the  amount  of  time  trainees 
at  each  of  six  hypermedia  workstations  spent  actually  interacting  with  the  hypermedia  training  materials  and 
the  total  training  time.  The  amount  of  time  trainees  spent  actually  interacting  with  the  hypermedia 
materials  was  surprisingly  low,  ranging  from  no  time  spent  using  hypermedia  software  by  one  trainee  on 
one  day  to  a  high  of  20%.  The  overall  mean  usage  time  for  all  trainees  over  the  first  four  days  of  training 
was  8%. 


Table  1 

Amount  of  Time  Hypermedia  Computers  Were  Used 
as  Percentage  of  Total  Training  Time 


Trainee  No. 

DSYl 

Day  2 

Dav  3 

Day  4 

Mean 

Std,  Dev. 

1 

9% 

18% 

3% 

6% 

9% 

7% 

4 

17% 

10% 

12% 

4% 

11% 

5% 

5  &  10 

12% 

6% 

10% 

4% 

8% 

4% 

6&9 

20% 

9% 

9% 

4% 

11% 

7% 

8 

16% 

2% 

7% 

3% 

7% 

6% 

11 

11% 

0% 

6% 

3% 

5% 

5% 

Note  1.        Day  five  of  training  was  a  half-day  session  devoted  to  trainees'  presentation  of  action  plans 
and  course  evaluation.  Only  one  trainee  used  the  hypermedia  on  day  5  for  eight  minutes. 
Therefore,  analysis  of  day  5  usage  was  dropped. 

Note  2.        Overall  mean  =  8%;  S.D.  =  6%. 


Table  2  presents  the  beginning  and  ending  times  of  interactions  with  the  hypermedia  engaged  in 
by  trainees  during  the  first  four  days  of  training.  This  table  presents,  in  tabular  form,  a  picture  of  the  usage 
patterns  of  trainees. 
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Table  2 

Usage  Patterns  of  Hypermedia  Computers  During  Training 


Trainee 

Day. 

Start  1 

End  1 

Start  2 

End  2 

Start  3 

End  3 

Start  4 

End  4 

Start  5 

End  5 

1 

1 

11:07 

11:41 

12:46 

12:53 

1:05 

.1 

2 

10:20 

10:57 

4:07 

4:18 

4:51 

4:56 

5:30 

5:43 

5:56 

5:58 

3 

9:50 

9:59 

10:11 

10:12 

11:03 

11:04 

4 

9:40 

9:41 

10:50 

11:11 

4 

1 

11:05 

11:44 

2:43 

3:09 

4:47 

4:58 

5:09 

5:11 

5:46 

5:56 

2 

10:24 

10:53 

1:33 

1:35 

3 

11:53 

12:06 

12:12 

12:47 

4 

10:53 

11:12 

12:41 

5&10 

1 

10:59 

11:34 

2:34 

2:58 

2 

10:12 

10:33 

3 

10:15 

10:22 

10:36 

10:37 

12:01 

12:35 

4 

10:44 

11:03 

6&9 

1 

11:04 

11:41 

12:18 

12:38 

2:40 

3:06 

2 

10:19 

10:55 

3 

12:09 

12:45 

4 

10:50 

11:09 

8 

1 

11:01 

11:34 

2:34 

3:00 

4:38 

4:54 

2 

i .  on 

1:39 

1 :53 

3 

9:33 

9:34 

12:03 

12:33 

4 

10:49 

11:04 

11 

1 

10:58 

10:59 

2:40 

2:54 

4:39 

4:54 

5:39 

6:03 

2 

3 

12:05 

12:11 

12:29 

12:34 

4 

10:43 

10:54 

11:15 

11:16 

Relationships  among  use  of  hypermedia,  use  of  non-hypermedia  activities,  and 
attainment  of  specific  learning  objectives.  Unfortunately,  this  portion  of  the  study  turned  out  to 
be  the  least  effective  of  the  entire  study.  The  major  problem  which  hindered  collection  of  comprehensive 
and  valid  data  relative  to  the  attainment  of  learning  objectives  was  that  the  trainers  varied  widely  in  the 
degree  to  which  they  attended  to  the  learning  objectives  specified  in  the  trainer's  resource  book  for  each 
learning  activity,  and  the  extent  to  which  they  carried  out  systematic  application  of  the  suggested  evaluation 
strategies.  On  some  occasions,  trainers  presented  the  learning  objective  for  a  learning  activity  they  were 
facilitating,  and  reminded  trainees  that  the  they  could  remind  themselves  of  the  learning  objective  by 
clicking  on  the  "Objectives"  icon  which  was  always  present  on  the  computer's  screen.  Other  trainers  did 
not  present  the  objective  explicitly,  and  in  some  cases  their  behavior  indicated  that  they  had  not  understood 
or  were  not  aware  of  the  learning  objective.  Applications  of  the  evaluation  strategies  was  sporadic  at  best. 
For  these  reasons,  it  was  not  possible  to  continue  the  analysis  of  the  relationship  between  uses  of 
hypermedia  and  the  attainment  of  specific  learning  objectives. 


Analysis  of  hypermedia  paths  used  by  trainees.  Data  on  the  type  of  hypermedia  paths 
used  by  various  trainees  are  in  the  process  of  being  analyzed.  Preliminary  analysis  of  the  d£ta  indicates  that 
trainees  often  traveled  extremely  complex  paths  while  using  the  hypermedia  training  materials. 
Categorization  of  these  paths  has  proven  difficult  and  time-consuming.  A  number  of  tools  are  under 
development  to  facilitate  categorization  of  hypermedia  paths.  These  tools,  and  the  results  of  their 
application,  will  be  described  in  a  future  paper. 


Discussion 


The  volume  of  information  collected  through  the  qualitative  component  of  the  study  proved 
challenging  to  summarize  and  comprehend.  The  results  of  the  data  analysis  process,  however,  met  the  goals 
of  the  research  study,  in  that  a  clear  and  comprehensive  picture  of  what  happened  during  a  five-day 
hypermedia  training  program  was  developed.  The  comprehensive  picture  provided  a  foundation  for 
identifying  a  number  of  important  questions  which  are  appropriate  for  follow-up  study.  Several  of  these 
questions  can  be  expressed  as  null  hypotheses  which  can  be  investigated  by  applying  quasi-experimental 
techniques.  The  qualitative  research  has  provided  the  context  and  laid  the  groundwork  for  more  purely 
quantitative  research. 

Although  the  qualitative  data  had  indicated  that  trainees  spent  relatively  little  time  actually 
interacting  with  the  hypermedia  software,  the  picture  of  usage  provided  by  the  computer  tracking  date  was 
surprisingly  low.  An  overall  mean  usage  time  for  all  trd-iees  over  the  first  four  days  of  training  of  8% 
could  indicate  that  trainers  were  not  encouraging  trainees  to  make  appropriate  use  of  the  hypermedia. 
Qualitative  data  indicated  that,  in  several  instances,  trainers  presented  content  identical  to  that  included  in  the 
hypermedia  using  overheads,  flip  charts,  and  lecture.  On  the  other  hand,  it  should  be  kept  in  mind  that  the 
hypermedia  training  materials  studied  were  designed  as  integrated  hypermedia  to  be  used  to  support 
interpersonal  interactions  (role  playing,  small  and  large  group  discussions,  etc.)  which  clarified  and  extended 
the  information  presented  in  the  hypermedia.  Follow-up  activities  of  this  type,  whether  appropriate  or 
inappropriate,  would  not  show  up  in  the  tracking  data  as  time  spent  using  the  hypermedia,  even  though  the 
activities  clearly  were  extensions  of  the  learning  begun  through  interaction  with  the  hypermedia  materials. 

Analysis  of  data  generated  by  the  computer  tracking  component  of  the  study  proved  more 
challenging.  The  body  of  information  is  extremely  large  and  extremely  complex,  particularly  in  regards  to 
the  depth  and  variety  of  hypermedia  paths  traversed  by  users.  The  flexibility  provided  by  hypermedia 
materials  clearly  has  both  positive  and  negative  aspects.  In  order  to  identify  these  aspects,  and  to  optimize 
the  use  of  hypermedia  materials,  it  is  necessary  first  to  develop  a  taxonomy  of  types  of  paths  which  are  used 
by  different  trainees.  The  relationships  among  types  of  paths,  learning  styles,  and  learning  outcomes  can 
then  be  investigated  more  systematically.  One  promising  approach  to  developing  such  a  taxonomy  would 
be  to  examine  the  techniques  developed  by  social  scientists  to  categorize  and  analyze  interactions  in  small 
groups. 

The  inability  to  formally  examine  the  relationships  among  uses  of  hypermedia,  uses  of  other  non- 
hypermedia  training  activities,  and  the  attainment  of  specified  learning  objectives  was  disappointing  to  the 
researchers.  The  researchers  were  also  key  members  of  the  team  which  had  designed  and  developed  the 
hypermedia  training  course,  and  had  worked  diligently  to  develop  meaningful  objectives  and  appropriate 
evaluation  strategies  for  these  objectives.  In  addition,  the  researchers  were  keenly  aware  of  the  importance 
placed  on  achieving  specific  learning  objectives  by  the  Cities  in  Schools  administrators  who  coordinated  the 
CIS  hypermedia  development  project.  Cities  in  Schools  had  invested  considerable  resources  in  the 
hypermedia  training  curriculum,  and  it  was  important  to  demonstrate  that  these  resources  had  been  invested 
wisely. 


Conclusions 


This  qualitative  study  of  a  hypermedia  training  program  treated  a  particular  group  of  trainees  and 
trainers  engaged  in  a  particular  training  program  as  a  "culture"  to  be  investigated  through  intensive  and 
careful  observation.  The  data  gathered,  as  in  most  qualitative  studies,  is  voluminous,  and  requires  extensive 
and  careful  analysis  in  order  to  extract  useful  patterns  and  to  draw  meaningful  conclusions.  Since  each 
rroup  of  trainees  and  trainers  and  each  training  program  application  is  different,  it  is  not  appropriate  to 
extrapolate  the  results  of  this  study  to  all  training  programs  using  hypermedia  application. 
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Qualitative  studies  are,  however,  extremely  useful  for  providing  a  comprehensive  picture  of  what 
actually  happens  in  a  "real-world"  training  situation.  This  picture  provides  a  solid  foundation  on  which  to 
develop  questions  for  further  study,  which  can  then  be  investigated  systematically  using  quasi-experimental 
methodologies. 

Computer  tracking  .ools,  to  be  implemented,  require  considerable  programming  expertise  and 
effort.  In  addition,  they  are  most  efficiently  implemented  if  they  are  incorporated  into  the  design  of  the 
hypermedia  application  and  implemented  as  the  hypermedia  application  itself  is  developed.  Difficulties 
inherent  in  allocating  scarce  project  resources  to  develop  tracking  tools,  including  the  challenge  of 
incorporating  tracking  tools  while  meeting  pressing  delivery  due  dates,  makes  it  difficult  to  implement 
tracking  tools  on  a  wide  basis.  In  addilion,  it  is  extremely  difficult  to  add  tracking  tools  on  a  post  hoc 
basis,  especially  by  people  other  than  the  group  who  originally  designed  and  developed  the  hypermedia 
application.  For  these  reasons,  tracking  tools  have  not,  to  date,  been  used  extensively  in  hypermedia 
research. 

These  researchers  believe  that  computer  tracking  tools  hold  great  promise  for  providing  insights 
into  how  hypermedia  applications  are  actually  used  and  how  they  can  be  used  more  effectively.  These  tools 
are  particularly  effective  when  their  use  is  combined  with  other  research  methodologies,  such  as  qualitative 
investigations  of  specific  hypermedia  training  courses.  Both  qualitative  research  methodologies  and 
computer  tracking  methodologies  hold  great  promise  for  furthering  understanding  of  how  hypermedia  can  be 
designed  for  effective  training.  Both  have  the  advantage  of  being  applied  in  "real-world"  settings,  studying 
actual  applications  of  hypermedia  as  they  are  actually  being  used  without  distorting  that  application  by  their 
presence.  When  used  in  combination,  qualitative  research  methodologies  and  computer  tracking 
methodologies  provide  a  synergistic  extension  of  the  information  provided  by  each,  and  therefore  provide  a 
comprehensive  and  powerful  foundation  on  which  to  base  future  decisions  about  the  design  and  delivery  of 
hyperrnedia-based  training. 
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Introduction 


The  typical  class  in  almost  every  educational  situation  (K-12  as  well  as  most  graduate  education)  is 
measured  in  terms  of  contact  time  i.e.  the  time  the  teacher  is  interfacing  with  the  students.  Computer 
Mediated  Communication  involves  the  use  of  the  computer  to  initiate  and  respond  to  issues  via  an  e-mail 
software  application  in  conjunction  with  a  network  connection.  The  advantage  of  this  communication 
modality  is  that  participants  can  be  and  usually  are  separated  by  space  and  time.  This  means  that  student 
questions  and  concerns  about  erudite  concepts  recently  encountered  during  study  or  reflection  of  class 
discussions  can  be  addressed  while  the  issue  is  fresh  in  the  student's  mind. 

Computer  Mediated  Communication  (CMC)  can  increase  contact  time  as  well  as  provide  an  opportunity  for 
the  student  to  address  issues  to  the  teacher  and  his/her  peers  after  the  concept,  issue  or  problem  is  presented. 
CMC  also  enables  the  student  to  practice  developing  their  own  communication  styles  using  the  instructors 
as  consultant/advisors  (Philips  and  Santoro,  1989).  Perhaps  the  most  powerful  advantage  of  CMC  is  that  it 
provides  the  opportunity  for  students  and  faculty  to  clarify  concepts.  The  use  of  CMC  in  a  course  provides 
for  discursive  feedback  stripped  of  all  the  roles,  gestures,  body  language,  and  other  nonverbal  forms  of 
interaction  that  color  meaning.  In  addition,  CMC  allows  the  students  to  see,  reflect,  ruminate,  refute  and 
structure  the  CMC  discussion  points.  It  makes  the  discussion  tangible  by  making  it  visual.  Small 
conceptual  nuances  can  be  missed  in  discussions  infused  with  social  and  personal  contexts.  Viewing  the 
argument  makes  the  concept  concrete  for  student  review  with  CMC. 

Finally,  CMC  allows  tracking  of  the  concept  discussions  in  a  historical  or  chronological  fashion.  This 
can  also  be  a  disadvantage  as  communications  are  received  in  a  chronological  order  and  a  student  signing  on 
to  e-mail  tends  to  respond  to  the  questions  in  the  order  they  are  received.  If  a  student  does  not  stay  current 
on  a  daily  basis  with  the  communications  he/she  may  find  themselves  well  behind  in  the  discussion.  The 
student  needs  to  be  engaged  in  the  activity.  Scardamalia  and  Bereiter  termed  this  engagement 
"intentionality"  and  has  identified  it  as  an  important  variable  in  the  use  of  Computer  Mediated 
Communication  (Scardamalia,  M.,  &  Bereiter,  C.  1989).  This  engagement  or  "intentionality"  is,  of 
course,  important  for  all  learning. 

Factors 

Factors  that  facilitate  the  use  of  Computer  Mediated  Communication  in  decision  making  situations  are  well 
established.  McLeod  states  that  computer  mediated  communication  systems  typically  increase  decision 
quality ,  time  needed  to  make  a  decision,  equality  of  participation,  and  degree  of  task  focus.  However, 
studies  also  show  that  CMC  in  decision  making  decreases  consensus  and  satisfaction  (McLeod,  1992). 
These  same  factors  are  important  in  the  learning  situation.  The  frequency  of  participation  in  small  groups 
is,  however,  related  to  the  social  context,  group  composition,  role  of  the  leader,  norm  regarding  student 
participation  and  the  goal  of  the  meeting  (Stephen  &  Mishler,  1952).  No  studies  currently  address  the  issues 
of  the  level  of  the  debate  or  the  effect  of  computer  mediated  communication  on  the  group  interaction.  Issues 
of  quality  of  interaction  and  contribution  are  more  related  to  group  feeling.  The  major  premise  of  this  study 
is  that  having  students  engage  in  computer  mediated  communication  will  increase  the  cognitive  level  of 
student  discourse  by  allowing  student  to  reflect  on  difficult  concepts  on  an  as  needed  basis.  In  addition, 
this  e-mail  interaction  will  produce  a  positive  group  feeling  and  movement  of  the  class  toward  closer 
personal  relationships.  Lewin  identified  several  factors  which  impact  on  the  group  dynamic  and  can  be 
applied  to  the  analysis  of  CMC  interactions.  He  elaborated  these  dynamics  as  "field  theory." 

Field  theory  considers  components  of  group  interaction  based  on  several  factors.  These  factors  include 
spatial  factors,  systems  of  tension  and  field  forces.  Spatial  factors  are  geometric  adaptations  of  space  of  free 
movement,  both  physically  and  intellectually.  The  second  class  of  concepts  is  anchored  in  dynamic 
psychology  of  the  individual  (e.g.,  need,  aspiration  level,  satiation).  These  latter  concepts  for  the  most  part 
refer  to  systems  of  tension  within  the  person  himself.  Pressures  within  the  individual  and  pressures 
emanating  from  the  surrounding  field  are  termed  field  forces  (motives  clearly  depending  upon  group 
pressures)  barriers  (obstacles  to  individual  action  owing  to  group  restraints),  or  locomotion  (changing  of  the 
individuals  position  with  reference  to  the  group)  (Allport  in  Lewin,  48).  Lewin  suggests  that  the  group 
dynamics  are  controlled  by  the  balance  between  these  forces.  By  extension,  analyzing  the  group  for  these 
perspectives  will  aid  in  understanding  both  the  dynamics  and  productivity  of  the  group.  This  study  is 
primarily  based  on  these  factors. 
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Field  Theory 


Field  theory  is  a  conceptual  framework  for  modeling  the  "psychological  forces"  influencing  individuals  at 
any  given  instant.(Diamond,  92).  According  to  Lewin  (1942)  there  are  six  attributes  of  field  theory  that  are 
particularly  important: 


1 .  The  use  of  a  constructive  method 

2 .  A  dynamic  approach 

3.  An  emphasis  upon  psychological  Processes 

4.  Analysis  based  upon  the  situation  as  a  whole 

5.  A  distinction  between  systematic  and  historical  issues 

6.  A  mathematical  representation  of  psychological  situations. 


Lewin  used  these  attributes  to  analyze  and  describe  group  dynamics  in  many  situations.  A  short  discussion 
of  these  attributes  follows. 

In  the  constructive  approach  elements  are  grouped  according  to  their  relationships.  The  elements  we  will 
consider  are  "life  space",  "field  forces"  and  "tension  systems"(figure  1).  The  life  space  of  an  individual 
consists  of  the  person  and  the  psychological  environment  that  exists  for  him  or  her.  Behavior  according  to 
Maruyama(1992)  is  a  function  of  that  person  and  that  person'  environment.  According  to  Shaw(1982),  it  is 
the  totality  of  all  psychological  factors  that  influence  the  individual  at  any  given  moment.  In  a  similar 
manner,  the  life  space  of  a  group  consists  of  the  group  and  the  environment  as  it  exists  for  the  group  at  the 
time.  Therefore  the  behavior  of  any  individual  or  group  is  a  function  of  the  person/group  and  the 
person/group  environment.  Individual  life  space  is  differentiated  into  regions  and  dependent  on  unique 
variables  of  the  individual.  It  has  two  additional  dimensions. 


Environmental 

Social 

Personal 


  Life  space 

figure  1 


The  first  is  the  fluidity  of  the  system  which  is  the  degree  to  which  regions  are  distinguishable  from  each 
other  and  its  reality/irreality  characteristic.  If  the  boundaries  are  rigid  and  very  different  then  fluidity  or  ease 
of  movement  from  on  region  to  the  next  is  limited.  This  tends  to  force  choice. 

The  second  is  the  level  of  irreality  whi>;h  involves  imagery  and  fantasy,  whereas  the  level  of  reality  involves 
more  objective  aspects  of  the  life  space.  The  level  of  reality  in  the  psychological  future  corresponds  to 
what  is  expected,  whereas,  the  level  of  irreality  corresponds  to  the  fears  for  the  future.  Dynamically,  the 
level  of  irreality  is  more  fluid  and  more  closely  related  to  the  central  layers  of  the  personality  that  the  level 
of  rea!ity(Shaw,  1982). 


a 

ERIC 


2i 


188 


Behavior  is  identified  as  the  movement  or  locomotion  of  the  person  in  life  spa-     There  are  different  types 
of  locomotion-  include  bodily  locomotion,  or  approach  or  avoiding  goals(psy ecological  locomotion). 
Locomotion,  according  to  Shaw,  may  be  produced  by  a  need,  which  corresponds  to  a  tension  system  in  the 
inner-personal  region.  The  extent  to  which  the  need  will  produce  locomotion  depends  in  part  on  the  degree 
to  which  the  inner-personal  region  is  in  communication  with  another  region,  where  two  regions  are  said  to 
be  in  communication  if  a  change  of  state  in  one  region  produces  a  change  of  state  in  the  other  region.  If 
two  regions  are  in  communication  and  a  need  is  aroused  in  one  of  the  regions,  locomotion  from  that  region 
to  the  second  occurs  until  a  state  of  equilibrium  is  reached,  that  is,  until  the  opposing  forces  in  the  two 
regions  are  equal  in  strength.(Shaw,  1982) 

A  force  is  defined  as  that  which  causes  change  (Lewin,  1936).  Its  properties  are  strength,  direction,  and 
point  of  application.  Thus  for  any  given  point  in  the  life  space,  the  construct  force  represents  the  direction 
of  and  tendency  to  change.  A  number  of  forces  can  act  on  the  same  point  at  any  given  time,  and  the 
combination  of  these  forces  is  called  the  resultant  force.  This  is  the  effective  force  operating  to  determine 
behavior  When  the  resultant  force  is  greater  than  zero,  there  is  either  a  locomotion  in  the  direction  of  that 
force  or  a  change  in  the  cognitive  structure  that  is  equivalent  to  locomotion.  Conversely,  if  a  locomotion 
or  change  in  structure  occurs,  resultant  forces  exist  in  the  direction  of  that  change  (Lewin,  1940).  1  he 
strength  of  force  toward  or  away  from  a  goal  is  a  function  of  the  strength  or  die  valence  and  the 
psychological  distance  between  the  person  and  the  goal. 
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Fiaure2(a)  A  force  field  corresponding  to  two  positive  valences:  (b)  a  force  field  corresponding  to 
two  negative  valences:  (c)  a  force  field  corresponding  to  a  positive  and  a  negative  valence  in  the  same 
direction. (Shaw,  82) 
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Lewin  identified  several  types  of  forces:  driving  forces,  restraining  forces,  induced  forces,  forces 
corresponding  to  one's  own  needs,  and  impersonal  forces.  Forces  toward  a  positive  or  away  from  a  negative 
valence  are  called  driving  forces  because  they  lead  to  locomotion  (figure  2).  However,  locomotion  may  be 
hindered  or  prevented  by  physical  or  social  obstacles  or  barriers.  Such  obstacles  are  called  restraining  forces 
They  do  not  lead  to  locomotion  but  they  do  exert  an  influence  on  the  effects  of  driving  forces  (Lewin, 

In  any  life  space  tensions  exist.  When  the  strength  of  one  set  of  tensions  is  stronger  than  another 
movement  or  locomotion  from  or  toward  that  region  may  be  instigated.  This  locomotion  results  in  change 
When  regions  are  highly  permeable  (i.e.  no  rigid  boundaries  between  regions)  the  change  can  be  systematic  ' 
across  several  regions.  These  concepts  will  be  used  to  interpret,  the  dynamics  observed  in  the  study. 

Each  student  has  their  own  life  space  and  interacts  with  other  student's  life  space  in  class  with  all  the  social 
trappings.  As  a  student,  each  is  in  a  competitive  environment  with  others  in  the  class.  A  goal  of  the  class 
is  construct  a  similar  mental  model  of  the  subject  matter  in  each  student.  This  is  traditionally  done  through 
lecture,  assigned  readings  and  classroom  discussion.  The  idea  is  to  get  the  student  to  move  cogni lively 
toward  this  mental  model  by  reducing  the  barriers  to  that  movement.  By  stripping  away  the  social 
environment  and  reducing  the  language/speaking  barrier  using  CMC  it  is  assumed  that  the  student  will  be 
freer  to  make  that  cognitive  movement  towards  the  mental  model  i.e.  making  one  set  of  tensions  stronger 
than  the  other.  In  addition,  the  time  to  reflection  will  result  in  the  student  creating  a  realistic  argument  for 
a  particular  set  of  beliefs  as  they  develop  their  conceptual  model.  Therefore,  the  student  will  tend  more 
toward  evaluating  the  concept  in  their  discussion  than  just  summarizing  others  thoughts. 


The  target  population  for  the  research  consists  of  seventeen  adults  enrolled  in  a  course  entitled,  "Designing 
Cognitive  Learning  Environments"  at  The  Pennsylvania  State  University  in  the  spring  of  1994.  The  goal 
of  the  proposed  project  was  to  research  the  effect  of  computer  mediated  communication  on  group  dynamics 
according  to  "field  theory." 

Discussing  class  concepts  via  e-mail  was  a  stated  requirement  of  the  course.  Students  were  encouraged 
during  regular  class  meetings  to  participate  and  discuss  issues  via  e-mail.  Interesting  discussions  were 
continued  during  the  regular  weekly  class  meetings.  Two  communication  packages  were  available  to  the 
students.  Pop-mail  which  provided  a  word  processing  type  interface  and  PSUVM  which  provided  more  of  a 
command  line  type  interface.  The  choice  of  communication  packages  was  left  up  to  the  s»  Jent  as  was 
getting  an  access  account. 

Two  questionnaires  were  administered  to  students.  The  first  questionnaire,  early  in  the  semester,  gathered 
data  concerning  hardware,  software,  existing  personal  relationships,  personal  data,  biographical  data,  type  of 
e-mail  communication  package,  access  to  computers,  types  of  computer  available,  location  of  access, 
problems  with  access,  and  spatial  relationships.  The  second  questionnaire  administered  at  the  end  of  the 
course  gathered  data  concerning  current  spatial  relations,  systems  of  tension  and  field  forces.  All  class 
communications  were  saved.  A  protocol  was  developed  to  analyze  the  communication  in  terms  of  the  level 
of  interaction,  direction  of  interaction,  and  communication  relationships.  Electronic  mail  communication 
was  analyzed  in  terms  of  criteria  adapted  from  the  cognitive  activity  levels  as  discussed  by  Jonassen  in 
Mindtools  (Jonassen,  94). 
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Jonassen  identified  nine  cognitive  activity  levels.  These  levels  are  evaluating,  analyzing,  connecting, 
elaborating,  synthesizing,  imagining,  designing,  problem  solving  and  decision  making.  These  terms  are 
defined  by  Jonassen  as  follows: 


Evaluating 

assessing  information,  determining  criteria,  prioritizing,  recognizing 
fallacies,  verifying 

Analyzing 

recognizing  patterns,  classifying,  identifying  assumptions,  identifying 
main  ideas,  finding  sequences 

Connecting 

comparing/contrasting,  logical  thinking,  inferring  deductively,  inferring 
inductively,  identifying  casual  relation!  is 

Elaborating 

expanding,  modifying,  extending,  shifting  categories,  concretizing 

Synthesizing 

analogical  thinking,  summarizing,  hypothesizing,  planning 

Imagining 

fluency,  predicting  speculating,  visualizing,  intuition. 

Designing 

imagining  a  goal,  formulating  a  goal,  inventing  a  product,  assessing  a 
product,  revising  a  product. 

Problem  solving 

sensing  the  problem,  researching  the  problem,  formulating  the 
problem,  finding  alternatives,  choosing  the  solution,  building 
acceptance 

Decision  making 

identifying  an  issue  generating  alternatives,  assessing  the  consequences, 
making  as  choice,  evaluating  the  choices. 

Since  the  purpose  of  the  study  is  to  identify  higher  levels  of  cognitive  activity  due  to  reflection  the 
categories  of  problem  solving  and  decision  making  were  discarded  as  irrelevant  to  the  study. 

Analysis 

A  questionnaire  was  given  to  the  students  participating  in  the  class.  The  questionnaire  was  administered  at 
the  last  class  meeting.  The  data  collected  included  the  following: 

The  quality  of  interaction  each  student  had  with  their  classmates  at  the  beginning  of  the  semester 
and  the  quality  of  the  interaction  at  the  end. 

•  Experience  using  the  listserve. 

•  The  perceived  effects  the  conferencing  bad  on  student  learning. 

The  perceived  effectiveness  of  the  experience  in  facilitating  learning. 

•  The  frequency  of  signing  onto  the  listserve. 

What  the  student  did  with  the  messages  they  received. 

•  Did  they  read  the  messages  on-line? 

Self  ranking  on  a  Introvert  Extrovert  scale. 

Self  ranking  on  reflective  or  active  learning  style. 

In  addition,  class  electronic  communications  were  collected  by  listing  out  all  the  communications  on  the 
listserve.  These  were  organized  according  to  level  of  interaction  and  communication  pattern(i.e..  who  was 
involved  in  the  discussion). .Data  collected  from  the  questionnaire  were  entered  into  a  database  along  with 
the  abstract  rankings  of  the  messages  and  frequency  of  interaction.  Correlational  analyses  were  conducted. 
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Discussion 

During  the  fifteen  weeks  of  the  course  178  messages  were  generated  for  discussion  by  the  students.  Seven 
separate  themes  were  discussed  in  the  context  of  "constructivism".  The  length  of  the  messages  varied  as  did 
the  complexity.  The  messages  were  sorted  by  the  major  theme.  These  were  then  evaluated  by  several 
independent  evaluators  as  falling  into  one  of  the  categories  of  cognitive  activity  as  outlined  by  Jonassen. 
Chatty  messages  that  did  not  fall  into  any  of  the  categories  were  labeled  as  Not  Applicable.  The  message 
content  was  distributed  as  follows: 


Evaluating  14 

Analyzing  41 

Connecting  30 

Elaborating  17 

Synthesizing  51 

Imagining  09 

Designing  02 

Not  Applicable  14 


The  discussion  items  tended  io  move  up  the  rating  scale  as  the  student  became  an  expert  in  dealing  with  the 
issues  over  time.  This  trend  would  seem  to  support  the  contention  that  the  level  of  debate  was  rising 
throughout  the  term.  However,  it  is  not  clear  what  role  using  CMC  had  in  this  trend. 

The  students  overcame  significant  barriers  to  using  the  interface  effectively.  At  least  two  different  e-mail 
systems  were  available  for  students.  These  had  very  different  user  interfaces  with  one  being  difficult  to  use. 
The  other  was  more  forgiving  and  utilized  the  standard  features  of  a  word  processor.  Several  students 
indicated  that  they  had  trouble  getting  on  to  the  e-mai!  system  from  home  and  several  could  only  read 
messages  at  the  university.  Other  difficulties  mentioned  included: 

•  difficulty  down  loading  messages  to  disk 

•  getting  a  hard  copy  of  the  messages 

•  jumping  into  the  middle  of  a  discussion  to  make  a  point  when  the  conversation  has  moved  beyond 
that  point 

•  too  many  messages 

•  don't  have  enough  time  to  read  the  messages 

Students  indicated  that  they  felt  that  using  CMC  as  an  out  of  class  activity  enhanced  information  exchange 
and  gave  them  time  to  reflect  on  the  issues  under  discussion.  They  also  stated  that  it  gave  them  time  to 
resolve  issues  prior  to  attending  class.  Students  were  also  able  to  go  back  to  their  resources  and  reinforce 
the  argument  that  they  were  making  in  class.  It  also  gave  them  more  concrete  information  about  what 
other  students  were  thinking  about  the  subject.  CMC  argument  were  generally  more  organized  and 
referenced  than  the  oral  arguments  in  class. 

Most  participants  accessed  their  mail  on  a  daily  basis,  but  were  more  likely  to  respond  to  the  discussions  on 
the  day  of  the  class.  This  proved  to  be  a  disadvantage  for  some  students  as  they  may  not  have  seen  some  of 
the  discussion  points  before  they  arrived  at  class.  An  interesting  side  bar  was  that  most  of  the  participants 
did  not  read  the  messages  on-line,  but  rather  printed  them  out  before  reading  them.  So  much  for  the  cost 
savings  in  paper.  Another  interesting  highlight  was  that  some  participants  would  create  their  responses  in  a 
word  processing  environment  and  transfer  it  to  e-mail  so  that  it  would  be  spell  checked  prior  to  being  sent. 

Most  students  began  the  semester  with  few  personal  associations  among  the  group  and  although  some 
group  work  was  required,  most  of  this  was  accomplished  via  the  computer.  It  was  assumed  that  group 
dynamics  and  a  feeling  of  personal  closeness  would  be  facilitated  by  CMC  discussion  of  issues  relevant  to 
the  participant.  Spatial  grouping  indicated  that  students  who  participated  in  the  electronic  discussions 
ranked  other  participants  as  closer  to  them  than  the  students  who  were  less  frequent  participants.  Students 
tended  to  elaborate  the  concepts  from  a  personal  perspective  drawing  upon  prior  knowledge  more  than  doing 
extended  readings  to  support  their  individual  viewpoints.  This  tendency  caused  more  advanced  students  to 
comment  on  a  lack  of  depth  to  the  CMC.    Students  who  took  an  adversarial  role  in  the  CMC  discussions 
tended  to  show  less  growth  on  the  group  dynamic  scale.  The  spatial  maps  also  indicated  that  several  group 
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members  were  not  engaged  in  the  CMC  discussions.  These  same  students  tended  to  be  less  likely  to 
participate  in  class.  Discussion  initiators  tended  to  be  the  least  experienced  in  the  content  area.  Those  more 
familiar  with  the  content  tended  to  contribute  by  connecting  the  issues  and  clarifying  discussion  points 
rather  than  initiating  the  discussions..  Students  with  language  difficulties(English  as  a  second  language, 
ESL)  tended  to  participate  at  a  higher  cognitive  level  using  CMC  than  in  class,  but  still  tended  to  be  among 
the  least  frequent  users. 

At  the  onset  CMC  discussions  messages  were  more  likely  to  fall  in  the  lower  levels  of  cognitive  activity 
scale  of  not  applicable,  designing,  imagining  and  synthesizing,  while  later  messages  were  in  the 
connecting,  analyzing  and  evaluating  categories.  This  indicates  that  some  raising  of  the  debated  did  occur. 
This  may  be  largely  though  the  mentoring  activity  of  the  professor,  who  would  respond  to  messages  by 
making  connections  for  specific  discussion  questions  and  not  just  a  result  of  have  the  CMC  function 
available  to  students.  Barriers  to  communication  were  also  reduced  by  the  professor's  frequent  reinforcement 
to  student  communications.  Conceptual  movement  for  students  involved  in  a  topic  was  also  evidenced  by 
the  quality  of  the  communication  change  from  lower  to  higher  cognitive  activity  levels. 

Conclusion 

Eighty-four  percent  of  the  students  felt  that  the  CMC  caused  them  to  move  in  a  positive  direction  in  the 
both  the  personal  and  cognitive  areas.  However,  requiring  students  to  discuss  topical  issues  after  class 
using  e-mail  as  a  CMC  device  is  an  insufficient  situation  for  increasing  critical  thinking  skills  and  affecting 
the  group  dynamic.  The  number  of  variables  affecting  student  behavior  are  numerous  and  overcoming 
barriers  to  personal  and  mental  growth  must  be  addressed  in  a  more  systematic  way.  Some  barriers  to 
movement  included:  the  CMC  interface,  time  to  read;  time  to  formulate  a  response  and  address  as  many  as 
twenty  separate  concepts  in  addition  to  all  the  other  class  responsibilities;  and  student  individual  differences. 
The  CMC  interface  can  be  a  considerable  barrier  to  personal  and  cognitive  movement. 
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Introduction 


Traditionally,  technology  classes  have  been  taught  in  lock-step  fashion,  moving  from  technology 
to  technology,  emphasizing  mastery  of  specific  skills.  While  this  method  has  proven  effective  -  at  least  in 
relation  to  short-term  use  of  the  technologies  --  we  were  interested  in  creating  a  different  learning 
environment  for  teaching  technology  to  teachers.  In  redesigning  the  course,  we  employed  a  problem- 
centered,  activity-based  approach  where  technologies  are  "anchored"  in  authentic  and  familiar  problems  in 
which  teaching  and  learning  occurs.  This  approach  is  one  based  on  the  view  of  an  open  learning 
environment  where  the  learners  have  direct  input  for  the  direction  of  the  course.  It  is  a  problem-centered, 
activity-based  approach  to  teaching  various  technologies  to  address  everyday  teaching  and  learning  problems. 

Teaching  the  technology  in  this  manner  creates  a  new  learning  environment,  with  different 
requirements  and  expectations  not  only  for  the  learner,  but  for  the  instructor  as  well.  In  this  paper,  the 
implications  of  teaching  a  course  in  this  manner  will  be  discussed.  An  overview  of  the  course  will  be 
provided,  including  the  theoretical  foundations  underlying  the  learning  environment.  Participants,  goals  and 
objectives,  as  well  as  requirements  of  the  course  will  be  presented.  Discussion  of  moving  theory  into 
practice  in  the  actual  implementation  of  the  course  will  be  explored.  Promises,  problems  and  issues  in 
relation  to  teaching  in  this  type  of  learning  environment  will  be  discussed,  both  in  relation  to  the  learner 
and  the  instructor. 

Background 

Technology  for  Teachers,  first  offered  in  the  spring  of  1994,  is  focused  on  creating  the  conditions 
under  which  people  typically  work  to  understand  everyday  things:  first  developing  a  need  to  learn,  then 
satisfying  the  need.  We  work  to  create  "needs  to  know"  by  referencing  issues  for  which  the  learner  has 
already  acquired  significant  expertise,  and  examining  how  technology  can  help  the  learner  address  these 
issues.  Several  factors  contributed  to  the  way  the  course  is  currently  taught.  These  include:  a  desire  to 
create  a  context  for  learning,  wanting  to  teach  the  way  people  learn,  learners  entering  the  classroom  with  a 
variety  of  experience,  a  different  view  of  "technology"  and  a  desire  to  create  a  "hands-on,  minds-on" 
environment  for  learning  technology. 

Creating  a  Context  for  Learning 

The  educational  community  has  seen  an  influx  of  technology  into  the  classroom.  However,  the 
technology  growth  has  not  been  paralleled  in  use  and  integration  of  technology  into  the  everyday  classroom 
experience  (Carr,  Novak,  &  Berger,  1992;  Glenn,  1993;  Handler,  1993;  Ingram,  1992).  Educators  continue 
to  fall  behind  the  technology  curve,  with  adoption  resistance  a  predominant  theme.  While  researchers 
working  in  teacher  education  have  noted  potential  causes,  ranging  from  lack  of  modeling  to  lack  of  skill,  a 
fundamental  component  in  the  lack  of  adoption  appears  to  be  the  lack  of  a  mental  model  for  integration  of 
the  technology. 

In  working  toward  integration,  we  moved  away  from  a  primary  focus  of  technology  skills  to  one 
of  technology  integration.  To  create  this  atmosphere,  the  course  centers  around  creating  a  context  for 
learning.  The  participants  gain  experience  in  the  educational  uses  of  "things"  such  as  computers,  interactive 
video  and  electronic  communication;  the  experience  is  gained  by  applying  use  of  the  technology  to  practical 
problems  encountered  in  everyday  teaching.  The  technology  is  not  presented  as  the  solution,  but  rather  a 
solution  -  a  way  to  make  processes  and  tasks  easier  for  the  teacher,  learner,  and  parents. 

Teaching  in  the  Way  People  Learn 

In  addition  to  creating  a  context  for  learning,  we  were  also  guided  by  a  desire  to  teach  the  way 
people  learn:  developing  a  need  to  know,  then  satisfying  the  need.  The  course  centers  around  problems  and 
issues  encountered  in  the  everyday  classroom.  The  participants  work  toward  solving  the  problems,  taking 
into  account  multiple  points  of  view  and  using  a  variety  of  technologies.  The  course  focuses  on 
empowerment  of  the  learner:  they  create  their  own  goals  and  define  the  problems  they  want  to  solve. 

Different  Levels  of  Experience 

The  participants  in  the  course  are  primarily  Pre-K  to  12  teachers.  While  the  learners  share  a 
common  interest  in  education,  their  experiences  with  technology  are  quite  diverse.  Some  enter  the 
classroom  with  little  to  no  experience;  others  with  considerable  experience  in  many  of  the  technologies 
discussed  in  the  course.  This  left  us  with  the  challenge  of  accommodating  a  variety  of  experiences,  working 
to  keep  one  group  from  feeling  overwhelmed  and  the  other  from  becoming  bored. 
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"Technology"  beyond  "Things" 

Another  factor  contributing  to  the  way  the  course  is  designed  is  a  broadening  of  the  view  of 
"technology."  The  course  focuses  on  emerging  visions  of  schooling,  teaching,  and  learning.  Included  in 
these  visions  is  a  \aried  role  of  technology  for  supporting  the  visions.  Technology  is  defined  more  broadly 
than  "things"  and  "media."  Technologies,  in  the  context  of  the  course,  are  defined  as  practical  methods  used 
to  accomplish  practical  goals  —  the  organized  means  used  to  accomplish  diverse  ends.  The  course  is  not  so 
much  about  computers  as  it  is  about  using  technologies  in  a  broader  sense.  While  some  approaches  to  the 
problems  presented  focus  on  how  the  technology  can  create  new  ways  to  accomplish  existing  goals,  others 
emphasize  new  ways  to  exploit  roles  that  have  traditionally  been  untapped. 

Creating  a  "Hands-on,  Minds-on"  Environment 

As  in  most  technology  courses,  there  is  a  focus  on  creating  a  "learning  by  doing"  environment.  To 
afford  the  learners  with  a  hands-on  opportunity,  the  course  takes  place  in  a  networked  lab  environment 
utilizing  Macintosh-based  technologies.  It  is  an  ideal  setting  for  learning  the  technologies:  having  several 
technology  tools  readily  accessible  makes  it  easy  to  demonstrate  ideas,  present  information,  tap  into 
telecommunications  capabilities,  and  naturally  integrate  technology  into  the  activities  of  the  course. 

The  lab  is  also  used  to  enhance  a  "minds-on"  environment.  The  learners  are  presented  with  a 
variety  of  technologies,  as  well  as  a  learning  environment  where  exploration,  inquiry,  and  discovery  are 
modeled  and  encouraged.  Use  of  the  network  is  an  integral  part  of  the  course.  From  the  first  class  meeting, 
learners  are  exploring  the  network,  discovering  where  tools  for  the  course  are  located  and  feeling  their  way 
around  the  new  environment. 

In  addition  to  the  factors  leading  us  to  design  Technology  for  Teachers  in  the  way  it  is  currently 
taught,  there  is  an  underlying  theoretical  foundation:  open-ended  learning.  The  theory  and  its  principle 
assumptions  will  be  discussed  in  the  next  section. 

Theoretical  Foundation:     Open-Ended  Learning  Environment 

Interest  in  "unleashing"  the  capabilities  of  technologies  to  create  learning  systems  that  differ  from 
traditional  directed  instruction  has  recently  grown.  In  addition  to  the  interest  in  integrating  technology,  we 
also  wanted  to  create  a  learning  context  couched  in  the  principles  of  open-ended  learning  environments 
(OELEs).  The  factors  influencing  the  design  of  Technology  for  Teachers  made  this  type  of  learning 
environment  a  viable  solution  for  meeting  the  requirements. 

Despite  the  growth  in  interest  in  technology-based  learning  environments,  we  were  not  generating 
a  new  learning  environment  free  from  controversy.  Considerable  disagreement  exists  related  to  the  viability 
of  open-ended  learning  and  the  systems  that  reportedly  facilitate  it.  Advocates  point  to  benefits  such  as 
increased  flexibility  and  utility  (Spiro  &  Jengh,  1990;  Spiro  &  Feltovich,  1991),  and  improved 
individualization  of  knowledge,  as  evidence  for  the  promise  of  technology  in  user-centered  learning 
(Cognition  and  Technology  Group  at  Vanderbilt,  1991).  Skeptics,  on  the  other  hand,  question  what  is 
actually  learned  via  open-ended  systems  suggesting  that  effectiveness  claims  may  prove  impossible  to 
validate  (Dick,  1991).  The  challenge  remains  for  bridging  the  divergent  views  on  the  topic. 

Defining  Open-Ended  Learning  Environments 

A  significant  problem  in  this  debate  has  been  the  vagueness  with  which  open-ended  learning  and 
open-ended  learning  environments  (OELEs)  have  been  characterized.  Contributing  to  the  vagueness  is  a 
lack  of  common  definition  or  understanding  of  open-ended  learning  and  OELEs.  Perhaps  the  best  way  to 
understand  open-ended  learning  is  to  contrast  it  with  directed  learning.  Directed  learning  involves  the 
systematic  acquisition  and  retention  of  externally-defined  knowledge  and  skills.  An  individual  is  a  successful 
directed  learner  when  he  or  she  is  able  to  utilize  the  strategies  and  features  provided  by  the  instructor  to 
acquire  prescribed  knowledge  and  skills  to  acceptable  levels. 

Open-ended  leaning  refers  to  processes  wherein  the  intents  and  purposes  of  the  individual  are 
uniquely  established  and  pursued.  It  involves  the  individual  determination  of  what  is  to  be  learned,  how  it 
is  to  be  learned,  when  (or  if)  learning  goals  have  been  met,  and  what  (if  any)  subsequent  steps  might  be 
taken.  Any  number  of  things  might  be  candidates  for  individual  learning;  the  intents  and  goals  of  different 
learner  would  likely  vary  substantially.  In  effect,  the  fundamental  difference  between  directed  and  open-ended 
learning  is  in  who  determines  what  is  to  be  learned  and  what  steps  are  taken  to  promote  learning,  including 
the  resources  consulted,  both  internal  and  external,  during  the  learning  process. 
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Cognitive  Principles  Underlying  OELEs 

The  assumptions  of  open-ended  learning  are  not  merely  cosmetically  or  semantically  different  from 
assumptions  of  traditional  instruction,  but  are  fundamental  shifts  in  how  the  learner,  knowledge,  and  the 
structure  of  the  environment  are  conceptualized.  While  significant  variations  among  open-ended  learning 
environments  are  apparent,  they  also  share  several  common  assumptions  which  are  manifested  either 
explicitly  or  implicitly  within  the  environment.  Several  critical  assumptions  and  accompanying  beliefs, 
adapted  from  previous  research  (Hannafin,  Hall,  Land,  Hill,  1994),  can  be  identified. 

Context  and  experience  are  critical  to  understanding.  Open-ended  learning  environments  view  the 
processes  of  learning  and  the  context  in  which  it  occurs  as  inextricably  tied.  Open-ended  environments 
embed  learning  activities  in  authentic  contexts  —  as  concrete  instances  reflecting  problems  to  be  solved  —  in 
order  to  foster  thinking  that  originates  from  personal  and  practical  experiences  (Roth  &  Roychoudhury, 
1993). 

Understanding  is  individually  mediated.  In  open-ended  learning  environments,  the  learner  judges 
what,  when,  and  how  learning  will  occur  --  beliefs  influenced  heavily  by  constructivists  who  emphasize  the 
importance  of  constructing  personal  meaning  (Guba,  1990).  Self-directed  learning  requires  the  learner  to 
assume  individual  responsibility  for  the  learning  process:  asking  relevant  questions,  pursuing  needed 
knowledge,  evaluating  learning  experiences,  and  so  forth.  Learning,  in  this  manner,  is  thought  to  be  less 
passive,  less  compliant,  and  less  dependent  on  external  direction  than  learning  from  traditional  instruction 
(McCaslin  &  Good,  1992).  The  challenge  comes  in  determining  how  best  to  support  and  guide  these 
highly  individualized  processes  without  imposing  unnecessary,  potentially  conflicting,  external  structures. 

Cultivating  cognitive  processes  is  often  more  critical  than  generating  learning  products.  Open- 
ended  environments  are  designed  to  support  higher-order  cognitive  skills  such  as  identifying  and 
manipulating  variables,  interpreting  data,  hypothesizing  and  experimenting  (Roth  &  Roychoudhury,  1993). 
Their  open-ended  nature  reduces  the  emphasis  from  specific  content  and  rule-based  knowledge  in  favor  of 
self-reflections  and  thought-based  action.  This  focus  requires  learners  to  use  cognitive  and  metacognitive 
strategies  extensively.  Perkins  (1993)  refers  to  the  notion  of  "executive  function"  in  describing  the  self- 
directed  process  of  making  decision,  monitoring  thinking,  and  constructing  understanding.  Open-ended 
environments  provide  opportunities  for  learners  to  attend  to  things  they  perceive  as  relevant,  decide  for 
themselves  and  evaluate  their  own  thinking. 

Understanding  is  more  vital  than  knowing.  Much  of  the  criticism  surrounding  traditional 
instruction  is  that  it  often  stresses  recalling  information  rather  than  understanding  complexities  (Spiro,  et. 
al.,  1991).  Open-ended  learning  environments  immerse  learners  in  experiences  that  foster  understanding 
through  extended  exploration,  manipulation  and  opportunities  to  "get  to  know"  an  idea,  rather  than  simply 
being  told  about  it  (Papert,  1993).  Interactions  emphasize  the  underlying  reasons  "why"  an  idea  or  concept 
exists  instead  of  simply  acknowledging  or  accepting  its  validity.  Understanding  is  augmented  by  personal 
experiences,  points  of  view,  and  cognitive  scaffolding. 

Qualitatively  different  learning  processes  require  qualitatively  different  methods.  The  goal  of  many 
open-ended  learning  environments  is  to  immerse  learners  in  rich,  cognitively-complex  experiences  by 
providing  various  tools,  resources,  and  activities  with  which  to  think.  OELEs  focus  on  problem-solving 
skills  in  authentic  contexts,  affording  opportunities  for  exploration  and  theory  building  (Schwartz  & 
Yerushalmy,  1987),  and  providing  multiple  ~  sometimes  complimentary,  sometimes  competing  - 
perspectives  (Language  Development  and  Hypermedia  Research  Group,  1992).  Gagne  (1985)  was 
instrumental  in  acknowledging  that  varied  types  of  learning  require  differential  instructional  goals  and 
activities.  Gagne  and  Merrill  (1990)  noted  that  conceptually  complex  learning  goals  are  often  difficult  to 
achieve  with  traditional  instructional  approaches.  Multiple  context,  purposes,  resources  and  representations 
of  knowledge  may  be  necessary  in  order  for  learners  to  engage  in  complex  and  ill-structured  learning  (Spiro, 
etal,  1991). 

Guided  by  the  theoretical  foundations  of  OELEs,  as  well  as  the  factors  motivating  us  to  seek  a  new 
way  to  approach  teaching  technology,  Technology  for  Teachers  was  created.  Unlike  other  applications 
involving  OELE  principles,  such  as  the  Jasper  series  at  Vanderbilt  (Cognition  and  Technology  Group  at 
Vanderbilt,  1992)  and  Intermedia  at  Brown  University  (Yankelovich,  et  al,  1988),  Technology  for  Teachers 
is  based  in  real-time,  involving  the  continuous  interactions  of  humans.  Specifics  of  this  implementation 
are  discussed  in  the  next  section. 


ERIC 


197 


Technology  for  Teachers:     OELEs  in  Action 


The  educational  climate,  in  combination  with  the  externally  defined  requirements  and  our  interests, 
made  the  design  of  Technology  for  Teachers  as  an  OELE  possible.  Components  of  the  course  will  be 
discussed  in  the  following  sections.  This  includes  a  general  overview,  participants,  goals  and  objectives, 
course  requirements,  and  grading  procedures. 

Course  Overview 

Technology  for  Teachers  attempts  to  create  the  conditions  under  which  people  typically  work  to 
understand  things:  developing  a  need  to  learn,  then  satisfying  the  need.  The  course  operates  under  several 
important  premises.  First,  within  educational  settings,  technologies  are  not  so  much  a  curriculum  as  sets 
of  tools  to  be  put  to  use  productively.  The  course  works  to  establish  what  those  productive  uses  might  be. 

Another  premise  is  that  productive  use  is  influenced  by  three  factors:  context,  audience,  and 
activity.  Problems  presented  for  solution  in  the  course  are  anchored  in  everyday  classroom  teaching  and 
learning  problems  for  which  various  technologies  can  be  employed  for,  and  with,  learners,  teachers,  and 
communities.  The  goal  is  not  to  become  an  expert  in  technology,  but  to  become  more  of  an  expert  in 
teaching  and  learning. 

Finally,  the  course  operates  under  the  assumption  that  teaching  specific  technologies  will  only 
enable  the  learner  to  accomplish  a  task  in  the  short  term.  Technological  advances  continue  to  move  forward 
at  an  exponential  rate;  present  applications  will  inevitably  be  replaced  by  "new  best  ways."  In  moving  the 
learner  to  a  point  of  long-term  usage  and  adoption,  developing  problem  solving  and  cognitive  strategies  for 
the  use  of  technology  are  highlighted. 

Participants 

The  students  enrolled  in  the  course  are  diverse.  Students  taking  the  course  represent  various 
backgrounds,  both  in  their  professional  interests  as  well  as  their  technology  expertise.  The  course  not  only 
attracts  pre-service  teachers;  it  also  draws  its  population  from  a  variety  of  disciplines  across  campus: 
accounting,  communications,  leisure  services,  and  library  science.  The  diversity  of  the  classroom  is  also 
substantiated  by  the  fact  that  it  is  a  cross-level  course,  including  both  undergraduates  and  graduates. 

Technology  experience  is  just  as  variable  as  the  professional  interests  of  the  students.  Most 
rt'idents  enter  the  course  with  a  moderate  level  of  technology  experience;  however,  there  are  students  that 
fail  oi,  the  far  ends  of  the  experience  continuum.  A  few  students  have  never  touched  a  computer  and  have 
typical  fears  of  the  technology;  others  have  extensive  experience,  but  need  the  credit  in  order  to  meet 
accreditation  regulations  (Technology  for  Teachers  Surveys,  1994). 

Goals /Objectives 

The  primary  goal  of  the  course  is  not  to  create  experts  in  technology,  but  to  assist  the  learners  in 
increasing  their  levels  of  expertise  in  teaching  and  learning  with  the  thoughtful  application  of  technology. 
The  problems  presented  in  the  course  are  anchored  in  classroom  teaching  and  learning;  problems  for  which 
various  technologies  can  be  employed  for,  and  with,  learners,  teachers,  and  parents.  Significant  focus  is 
given  to  the  learner's  individual  growth  as  they  work  on  various  activities  to  solve  common  educational 
problems. 

Specific  objectives  for  the  course  include: 

•  To  identify  problems  for  which  varied  technologies  offers  useful  alternatives  for  teachers, 
learners,  and  parents 

•  To  plan  and  develop  alternatives,  featuring  varied  technologies  for  varied  target  audiences 

•  To  develop  a  personal/professional  growth  plan  for  gaining  facility  in  the  uses  of  varied 
technologies 

•  To  promote  personal/professional  growth  by  increasing  both  the  range  of  technology 
alternatives  and  the  ability  to  address  problems  with  them 

Components,   Tools,  and  Techniques 

There  are  several  components  to  the  course,  both  in  terms  of  the  tools  and  techniques  used  in  the 
course  and  what  is  required  from  the  learners.  Each  of  these  are  discussed  in  the  following  section. 

Tools.  From  the  outset  of  the  course,  the  learners  are  provided  with  tools  to  assist  them  in 
working  in  the  learning  environment.  In  addition  to  a  syllabus,  the  learners  receive  a  project  documentation 
tool,  a  portfolio  assessment  methods  tool,  and  a  portfolio  growth  scale.  A  textbook  is  also  used  in  the 
course:  Lockard,  Abrams,  and  Many's  book,  Microcomputers  for  Twenty-First  Century  Educators  (1994, 
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3rd  ed.).  The  text  is  not  used  in  lock-step  fashion;  rather,  it,  like  the  other  resources  made  available  to  the 
leaner,  is  used  as  a  supplement  to  what  is  occurring  in  the  context  of  the  classroom  setting. 

Accessing  information.  Establishing  procedures  for  accessing  information  is  a  fundamental 
component  of  the  course.  "Accessing  information"  is  defined  in  a  broad  sense.  The  network  is  used 
extensively  to  access  the  tools,  as  well  as  other  documents  that  are  loaded  to  clarify  or  explain  technologies 
being  discussed  in  class.  The  learners  are  encouraged  to  use  electronic  mail  to  communicate,  both  with  each 
other  and  with  us.  Learners  are  also  encouraged  to  seek  information  from  each  other,  as  well  as  other 
resources  (such  as  the  lab  assistants,  or  their  school  media  specialist)  beyond  the  instructors. 

Problem-centered,  activity-based.    Another  technique  used  in  the  course  is  the  introduction 
of  technologies  through  problems.  The  course  addresses  problems  and  issues  encountered  in  an  educational 
context.  Learners  are  encouraged  to  consider  ways  technology  can  be  used  to  address  the  problems  and 
issues.  Throughout  the  semester,  the  students  are  presented  with  problems  and  issues  to  consider  in  relation 
to  solving  educational  problems.  While  a  focus  is  placed  on  solving  those  problems  and  issues  with 
technological  interventions,  the  learner  is  encouraged  to  expand  their  views. 

Overtime,  it  is  expected  that  the  students  in  the  course  will  begin  to  re-frame  the  problems  and  to 
consider  alternative  approaches  and  methods  for  defining  the  teaching-learning  partnership.  For  examrjlf . 
the  set  of  problems  presented  to  the  learners  in  a  unit  on  telecommunications  is:  "How  can  my  ki 
communicate  with  kids  from  other  cultures  or  countries?;  How  can  I  find  the  latest  information  on  a  unit  I 
will  be  teaching  next  week?;  and  How  can  parents  find  out  information  about  the  academic  calendar  or  other 
school  events?"  This  approach  to  viewing  problems  from  multiple  perspectives  takes  place  with  all  of  the 
technologies  introduced  in  the  course. 

Demonstrations  and  practice.  Following  the  establishment  of  a  problem-centered  context 
for  learning,  technologies  are  demonstrated  to  the  class.  This  takes  the  form  of  "mind  food:"  it  is  a  time  in 
the  class  session  when  we  present  different  ways  to  use  the  technologies.  We  demonstrate  a  variety  of 
things  that  can  be  done,  centered  in  different  problems. 

Learners  next  "walk-through"  a  hands-on  process  with  us.  This  usually  takes  the  form  of  a  step- 
by-step  demonstration.  Practice  time  in  the  class  is  also  provided.  Several  activities  are  demonstrated  and 
practiced  with  each  technology.  This  gives  the  learners  the  opportunity  to  experience  a  variety  of  ways  to 
use  the  technology,  thus  helping  to  expand  their  views  of  how  to  incorporate  the  technology  into  their 
learning  environments. 

Requirements 

Learners  solve  problems  both  as  members  of  groups  and/or  as  individuals.  Three  unit  projects  are 
being  used:  two  occur  throughout  the  semester;  a  final  project  is  due  at  the  end  of  the  course.  The  unit 
projects  are  designed  to  integrate  the  technologies  in  a  logical  fashion.  The  first  project  integrates  word 
processing  and  databases.  The  second  project  integrates  telecommunications,  desktops  publishing  and 
graphics. 

The  final  unit  covered  in  the  course  deals  with  emerging  technologies,  including  videodisks, 
interactive  video,  and  multimedia  applications.  The  final  project  is  geared  toward  the  learners  using  these 
technologies.  They  are  also  allowed  to  create  an  integrated  project  reflecting  all  of  the  technologies  covered 
throughout  the  semester. 

All  of  the  projects  are  turned  in  and  graded  in  a  portfolio  format.  We  make  use  of  the  network  and 
have  a  mechanism  in  place  that  enables  the  learners  to  turn  in  all  of  the  work  to  us  in  electronic  form.  The 
electronic  portfolios  are  evaluated  after  each  project  and  feedback  is  provided  to  the  learners. 

Presentations  of  the  work  that  is  being  completed  occurs  throughout  the  course.  This  includes 
demonstrations  of  completed  work,  as  well  as  works  in  progress.  Use  of  presentations  also  exposes  the 
learners  to  presentation  technologies,  including  LCD  panels  and  demonstration  programs  like  PowerPoint. 

A  final  requirement  of  the  learners  is  a  reflection  paper.  This  paper  is  due  at  the  end  of  the  course 
and  is  designed  to  encourage  the  learners  to  look  back  on  their  progress  in  the  course,  reflecting  on  what 
they  did  and  where  they  would  like  to  go  next.  The  paper  also  addresses  feedback  on  the  course  itself. 
Learners  are  encouraged  to  be  critical  and  give  suggestions  for  improvement. 

Theory  into  Practice:  Promises  and  Problems 

Teaching  technology  in  an  OELE  brings  with  it  several  promises  and  problems.  Surveys  from  the 
three  courses  taught  last  year  illustrate  both  the  positives  and  negatives  associated  with  OELEs  -  at  least 
from  the  perspective  of  the  learner.  Each  of  these,  as  well  as  our  perspectives  and  unresolved  issues,  is 
discussed  below. 
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Promises 

Teaching  technology  in  an  OELE  is  a  challenging,  and  in  retrospect,  very  rewarding  task.  Not 
only  did  we  see  the  classes  as  a  whole  develop  in  their  technology  skills,  but  significant  strides  in 
individual  growth  was  encountered  as  well.  Examples  include:  all  learners  reported  an  increase  in  the  use  of 
technologies;  while  over  50%  of  the  participants  said  they  started  out  feeling  confused  about  the 
technology,  all  were  in  the  high  comfort  zone  by  the  end  of  the  course;  and  the  majority  of  the  students 
reported  that  they  liked  to  work  individually  and  be  assessed  by  individual  growth  (Technology  for  Teachers 
Surveys,  1994). 

Direct  statements  from  the  learners  can  attest  to  the  positive  results  of  teaching  technology  in  an 

OELE: 

•  I  am  leaving  this  class  with  not  only  a  great  deal  of  knowledge  about  computers  and 
their  possibilities  than  I  had,  I  think  I  most  importantly  am  leaving  with  confidence 
in  myself  when  it  comes  to  computers  with[out]  the  intimidation  that  can  hold 
anyone  back  from  venturing  and  learning. 

•  My  life's  philosophy  is  if  you  can  find  the  resource,  then  you  can  solve  the  problem. 
Working  with  computers  is  an  ongoing  process;  everytime  I  sit  down  in  front  of  one,' 
I  learn  something  different. 

•  I  believe  that  the  most  valuable  thing  I  learned  from  this  class,  however,  was  how  to 
overcome  my  fear  of  making  mistakes. ...I  learned  to  trust  myself  a  little  bit  more  and 
to  not  feel  as  threatened  by  new  technological  development. 

•  I  am  not  as  intimidated  by  computers  as  I  used  to  be  and  I  believe  I  can  do  anything 
on  the  Mac  now  that  I  have  a  basic  understanding. ...The  main  thing,  though,  is  that 
not  only  can  I  do  a  lot  more  things  with  computers,  but  my  fear  of  using  computers 
is  gone. 

•  I  have  learned  to  look  around  and  ask  around  for  what  technological  resources  are 
available....  I  assumed  that  if  technology  wasn't  blatantly  there  in  front  of  me,  that  it 
wasn't  there. 

•  The  most  beneficial  part  of  this  call  to  me  attitudinal.  I  am  more  enthusiastic  about 
learning  new  technologies  rather  than  fearing  them.  Another  beneficial  aspect  of  the 
course  was  the  ability  to  tailor  the  technology  applications  to  something  relevant  to 
us  as  individuals.  Learning  through  experimentation  and  hands-on  exposure  proved 
very  effective. 

These  statements  from  the  learners  in  Technology  for  Teachers  clearly  reflect  their  enthusiasm  for 
the  course,  the  techniques  and  methodologies  that  were  used,  and  the  benefits  the  learners  felt  they  received 
in  learning  in  this  environment.  Getting  to  this  point,  however,  was  a  long  and  often  painful  process,  both 
for  the  learner  and  us.  These  factors  are  discussed  next. 

Problems  and  Issues 

Perhaps  the  biggest  problem  facing  all  of  the  participants  in  the  course  was  the  time  factor  in 
relation  to  how  long  it  took  the  learners  and  us  to  become  comfortable.  All  of  the  participants  in  the 
course  are  accustomed  to  working  and  learning  in  a  very  directive  and  highly  structured  learning 
environment.  Technology  for  Teachers  is  anything  but  directive  and  highly  structured.  In  fact,  one  of  the 
criticisms  raised  by  the  students  is  that  it  lacked  detail  and  specifics.  "One  of  the  major  problems  I  had  with 
this  course  [was]  that  everything  seems  rushed  and  I  was  not  comfortable  with  anything  I  turned  in  because 
of  the  pressure  to  complete  my  project  on  time"  (Technology  for  Teachers  Reflection  Papers,  1994). 

We  also  encountered  experiences  of  disorientation  and  disorganization.  Teaching  in  an  OELE  is 
quite  a  different  experience  from  teaching  in  a  more  directive  environment.  The  challenge  for  the  instructor 
is  one  of  how  best  to  support  and  guide  processes  that  are  highly  individualized  without  imposing 
unnecessary,  potentially  conflicting,  external  structures.  Adapting  to  the  demands  of  these  learning 
environments,  getting  past  the  initial  bump,  is  an  issue,  both  for  the  learner  and  the  instructor. 

Closely  tied  to  these  feelings  of  discomfort  over  the  learning  environment  is  grade  stress.  This  is 
well  documented  by  one  student  when  he  states,  "I  think  another  reason  it  was  tough  [the  course]  is  that  the 
format  is  very  free  so  a  person  doesn't  know  what  qualifies  as  "A"  work"  (Technology  for  Teachers 
Reflection  Papers,  1994).  Grade  stress  had  an  effect  on  several  aspects  of  the  course,  including  resistance  to 
the  learning  environment  and  identification  of  growth  by  the  students  --  their  primary  focus  was  on  the  "A," 
or  lack  thereof.  Helping  learners  to  overcome  this  stress,  enabling  them  to  move  into  an  assessment  realm 
of  growth  continues  to  be  an  issue  to  be  resolved. 
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The  variety  of  experience  levels  in  the  course  is  another  issue  that  must  be  resolved.  While  several 
benefits  were  gained  from  having  a  variety  of  experience  levels  in  the  course,  including  sharing  of  ideas 
among  peers  and  peer-tutoring,  learners  reported  that  varying  levels  of  experience  was  distracting  and  that 
they  worried  about  holding  back  the  class  or,  in  turn,  being  held  back  by  the  class  (Technology  for  Teachers 
Surveys,  1994).  One  learner  voiced  her  concern  in  the  form  of  a  question:  "...wondering  how  you  could 
separate  the  class  from  beginners,  intermediate  and  expert....  How  does  one  balance  a  class  like  this???" 
(Technology  for  Teachers  Reflection  Papers,  1994).  We  continue  to  seek  the  best  answer  to  this  query. 

Technology  for  Teachers  is  a  learning  environment  founded  on  the  principles  and  assumptions  of 
open-ended  learning  environments.  Initial  strides  in  creating  a  context  for  learning  based  on  this  theoretical 
foundation  have  proven  successful;  however,  much  work  remains  to  be  done  to  overcome  the  problems  and 
issues  associated  with  these  environments.  Continued  work  and  research  in  these  environments  is  critical  if 
they  are  to  continue  to  expand  and  grow.  It  is  only  though  this  continued  work  that  educators  can  hope  to 
reach  the  ultimate  goal:  creating  learners  rich  in  cognitive  and  problem-solving  strategies. 
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Instructional  designers  are  expected  to  familiarize  themselves  with  content  from  a  variety  of 
content  areas.  But  what  happens  when  the  content  is  highly  technical,  scientific  content  and  the 
instructional  designer  has  little  or  no  background  in  the  area?  Can  instructional  designers  lend  their 
expertise  to  curriculum  development  efforts  at  the  postsecondary  level  in  subject  areas  such  as  engineering, 
biology,  physics,  or  chemistry? 

Computer-based  instruction  for  higher  education  science  instruction  appears  to  be  growing.  More 
science  texts  in  university  bookstores  are  being  packaged  with  accompanying  softwai„.  The  National 
Science  Foundation  has  funded  several  curriculum  development  projects  around  the  country,  and  many  of 
them  are  computer-based  instructional  multimedia  projects.  The  Worldwide  Web  also  provides  access  to 
several  computer-based  instructional  modules  for  the  sciences. 

Computer-based  instruction  could  have  considerable  impact  on  improving  the  quality  of  science 
education.  Simulations  and  interactive  problems  provide  a  means  for  students  to  explore  scientific  concepts 
ar.d  experiment  without  the  expense  or  hazard  of  using  actual  materials.  One  of  the  most  powerful 
strategies  for  science  education  is  visualization,  the  process  of  animating  concepts  on  the  computer  screen 
that  are  too  small  or  large,  or  too  abstract  to  be  viewed  in  real  life.  Visualization  provides  a  means  to 
bridge  the  gap  between  abstract  processes  and  students'  prior  knowledge.  According  to  Hiroshi  Higuchi  & 
Eric  F.  Spina  at  Syracuse  University  (1993), 

"Many  undergraduate  students  have  difficulty  applying  the  basic  principles  learned  in 
engineering  problems,  such  as  those  encountered  in  design  courses  and  experimental 
analysis.  One  reason  for  this  shortcoming  is  that  undergraduates  often  do  not  achieve  a 
thorough  understanding  of  the  underlying  physical  principles  in  basic  engineering  courses. 
If,  however,  students  are  exposed  to  a  visual  representation  of  the  relevant  principle,  and 
are  encouraged  to  manipulate  and  study  the  image,  then  a  lasting  understanding  of  the 
principle  may  be  reached  -  one  based  more  on  physical  intuition  and  a  "feel"  for  the 
phenomenon  than  on  memorization." 

Many  of  the  computer-based  instructional  software  packages  for  science  on  the  market  today  do  not 
use  the  computer  to  its  full  potential,  or  use  the  medium  inappropriately,  for  example,  simulations  designed 
for  the  computer  when  the  use  of  actual  materials  in  a  lab  would  be  more  appropriate.  It  is  important  for 
software  development  efforts  in  the  sciences  to  focus  on  the  computer  as  a  tool  to  explain,  elaborate, 
clarify,  and  visualize  aspects  of  a  lesson  that  cannot  be  explained  as  well  on  paper,  or  through  a  lecture. 

Thus,  computer-based  instruction  for  the  sciences  is  an  area  that  could  benefit  from  the  expertise  of 
instructional  designers.  However,  there  are  several  issues  concerning  the  collaboration  between  content 
experts  and  instructional  designers: 

•  Can  instructional  design  theory  be  applied  to  higher  education  science  content  where  the 
instructional  designers  don't  understand  the  content? 

«     How  familiar  should  an  instructional  designer  become  with  the  content? 

•  How  can  effective  interactions  be  developed? 

•  How  much  time  should  the  content  expert  be  expected  to  put  in? 

•  Who  is  a  better  source  of  content  for  creating  interactive  material  -  the  content  expert  or  a 
novice  in  the  field?  Who  is  the  most  cost-effective  source  of  information? 

•  Who  could  serve  as  an  interface  between  the  content  expert  and  instructional  designer? 

This  paper  focuses  on  the  instructional  design  process  as  it  relates  to  the  development  of  computer- 
based  instruction  for  higher  education.  We  examined  (1)  the  instructional  design  process  and  lines  of 
communication  in  our  own  computer-based  project,  (2)  the  instructional  design  process  of  personnel 
working  on  similar  computer-based  projects,  and  (3)  existing  computer-based  software  for  higher  education- 
level  science  content. 
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Center  for  Intcrfacial  Engineering  (CIE)  Curriculum  Development  Project 


Background 

The  Center  for  Interfacial  Engineering  (CIE)  Curriculum  Development  Project  began  in  June  of 
1993.  Interfacial  engineering  is  a  new  cross-disciplinary  field  that  integrates  research  activities  in  aero, 
electrical,  chemical,  mechanical,  and  civil  engineering,  and  chemistry.  This  curriculum  development  project 
is  funded  by  the  NSF  and  represents  a  partnership  between  engineers  and  educators  to  design  and  produce 
computer-based  instructional  modules.  The  modules  have  been  used  in  conjunction  with  a  textbook, 
Fundamentals  of  Interfacial  Engineering. 


Each  module  will  have  5  tracks:  (1)  the  main  track  provides  the  content,  and  is  essentially  a  linear 
presentation  of  information.  At  certain  points  throughout  the  main  track,  the  user  may  jump  to  other 
"tracks"  to  explore  more  in-depth  information:  (2)  the  theoretical  track  provides  derivations  of  equations, 
(3)  the  experimental  track  provides  simulations  of  experiments  that  explain  how  the  information  presented 
in  the  main  track  was  obtained,  (4)  the  application  track  provides  real-world  examples  of  the  concepts  in 
everyday  life  or  industry,  and  (5)  the  interactive  track  provides  problems  and  simulations  that  the  user  works 
through. 

The  development  team  currently  consists  of  one  primary  and  one  secondary  content  expert,  a  full- 
time  project  manager/editor,  a  full-time  instructional  designer/graphic  artist,  and  a  half-time  instructional 
designer/programmer.  At  the  time  of  this  writing,  work  on  the  main  tracks  is  almost  completed,  and 
development  of  the  theoretical  and  interactive  tracks  has  just  begun. 

Development  Process 

When  the  project  began,  the  content  expert  had  completed  the  instructional  analysis,  and  had 
developed  storyboards  that  consisted  of  text  and  schematic  figures  drawn  on  transparency  film.  At  the  time, 
the  development  team  consisted  of  one  content  expert,  a  full-time  project  manager/editor,  one  half-time 
instructional  designer/graphic  designer,  and  one  half-time  instructional  designer/programmer.  The  initial 
development  approach  was  to  have  a  word  processing  specialist  transcribe  the  storyboards  into  the  authoring 
software,  and  then  pass  the  module  along  to  the  project  manager  to  edit  the  text  and  equations.  Next,  one  of 
the  instructional  designers  would  program  the  module  into  the  interface,  and  then  give  it  to  the  other 
instructional  designer  to  put  in  graphics  and  animations.  As  this  system  evolved,  problems  began  to 
develop. 

Content  Experts  and  Instructional  Designers 

The  working  relationship  between  content  experts  and  instructional  designers  doe?  not  have  a 
reputation  for  being  smooth,  and  there  are  many  stereotypes  about  each  other's  competence.  Stephen 
Lower,  a  professor  of  Chemistry  at  Simon  Fraser  University,  in  his  paper.  How  to  Make  Computer- 
Assisted  Instruction  Fail  (February,  1993),  states 
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"The  lessons  themselves  should  have  been  designed  by  Education  specialists,  who  are 
more  likely  to  employ  well-established  response-reinforcement  techniques.  For  example, 
Statement:  "Fire  engines  are  red."  Question:  "What  color  are  fire  engines?"  B.  F. 
Skinner  used  this  technique  with  great  success  to  train  pigeons,  who  are  evidently 
immune  to  boredom.  This  will  turn  away  all  but  your  most  unimaginative  drudges." 

The  instructional  designers  on  the  CIE  project  had  heard  that  this  attitude  was  common  in  the  field. 
And  initially  this  appeared  to  be  true,  because  the  content  expert  did  not  believe  that  instructional  design 
theory  was  applicable  at  the  postsecondary  level,  since  most  research  on  the  subject  dealt  with  instruction  at 
the  K- 12  level. 

As  the  instructional  designers  and  content  experts  continued  to  discuss  the  development  of  the 
content,  it  became  apparent  that  they  were  discussing  the  same  instructional  design  principles,  but  using 
different  terminology.  For  example,  the  instructional  designers  would  talk  about  "test  items"  and 
"evaluation"  and  the  content  expert  would  refer  to  these  concepts  as  "interactions."  Consensus  about 
instructional  approaches  improved  once  the  team  developed  a  common  vocabulary. 

Familiarity  with  the  Content 

Initially,  the  instructional  designers  believed  that  they  were  expected  to  become  completely 
immersed  in  the  content.  The  content  expert  knew  that  they  would  never  become  completely  competent 
with  the  material,  but  still  had  the  expectation  that  the  designers  would  be  able  to  take  the  content,  and, 
from  a  brief  explanation,  be  able  to  transform  the  storyboards  into  workable  modules. 

As  the  content  expert  and  instructional  designers  worked  together,  they  eventually  realized  that  the 
instructional  designers  would  never  understand  the  content  entirely,  but  could  understand  the  structure  of,  or 
relationships  within,  the  content.  For  example,  the  instructional  designers  could  never  understand  the  full 
nature  of  inverted  micelles,  but  they  could  recognize  that  they  represent  one  type  of  amphiphilic  aggregate. 
As  a  result,  the  development  process  became  a  more  collaborative  process,  and  the  linear  development 
approach  was  replaced  by  an  iterative  one. 

As  the  instructional  designers  spent  more  time  working  with  the  content,  they  became  more 
knowledgeable,  and  better  able  to  see  relationships.  Also,  they  became  more  familiar  with  the  conventions 
that  the  content  expert  used  in  the  storyboards.  For  example,  when  the  content  expert  drew  a  box  with  an 
arrow,  it  represented  a  beaker,  and  an  enlarged  view  of  what  was  happening  in  the  beaker  on  a  molecular 
level. 

Overlap  of  Roles 

Developing  a  module  requires  each  team  member  to  have  some  familiarity  with  programming, 
graphics,  editing,  etc.  Problems  arose  when  team  members  moved  into  another's  area  of  expertise.  For 
example,  the  editor  had  to  re-word  some  incorrect,  elaborate  text  passages  that  had  been  written  by  other 
team  members,  the  programmer  had  to  rewrite  unnecessarily  complicated  code,  and  the  graphic  artist  had  to 
recreate  inaccurate  graphics  and  animations. 

This  problem  has  not  been  completely  solved,  but  it  has  been  minimized.  In  most  instances,  the 
text  is  usually  changed  only  with  the  editor's  approval,  interactions  are  worked  out  on  paper  first  and  the 
content  expert  and  programmer  work  together  to  put  them  onto  the  computer,  and  graphics  are  either 
rendered  as  schematically  as  possible. 

Time  Demands 

Module  development  came  to  a  halt  when  there  were  unanswered  questions  about  the  content.  The 
development  staff  had  to  wait  (sometimes  for  weeks)  until  the  content  expert  could  find  the  time  to  explain. 
An  organizational  expert  on  staff  at  the  Center  was  asked  to  examine  the  team's  process  and  determine  ways 
to  make  it  more  efficient.  After  talking  with  the  team  and  drawing  a  graphic  representation  of  the  process, 
he  calculated  that  it  would  take  over  20  years  to  complete  the  project,  because  each  iteration  involved  the 
content  expert. 

To  solve  this  problem,  a  retired  professor  of  chemistry  was  contracted  part-time  to  serve  as  a 
secondary  content  expert.  This  solution  had  the  greatest  impact  on  the  project.  The  professor  was  able  to 
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answer  questions,  explain  processes,  create  charts  and  graphs,  and  edit  content  for  accuracy.  He  provided  a 
crucial  link  in  the  development  process,  as  he  had  the  necessary  scientific  background  and  the  time  to  devote 
to  the  project. 

Content  Accuracy 

The  instructional  designer/graphic  designer  would  experiment  with  different  ways  of  presenting  the 
content.  Most  of  the  time  the  approaches  were  successful,  and  the  rest  of  the  team  supported  her  creative 
license.  However,  on  a  few  occasions,  the  content  was  altered  to  the  point  of  inaccuracy,  much  to  the 
distress  of  the  project  manager.  The  project  manager  asked  the  instructional  designer  to  check  with  the 
content  expert  before  changing  anything,  but  the  instructional  designer  felt  that  doing  so  would  be 
excessively  inhibiting. 

After  discussing  the  problem  with  the  project  manager,  they  agreed  that  the  instructional  designer 
could  continue  to  experiment,  but  on  highly  technical  content  or  on  sections  containing  many  equations, 
the  instructional  designer  would  work  on  a  copy  of  that  section,  saving  the  original  approach.  When  the 
project  manager  comes  to  that  section,  he  can  consult  with  the  secondary  content  expert  and  make  the  final 
decision  about  which  part  to  retain. 


After  addressing  these  problems,  the  development  process  evolved  to  the  following: 


Storyboards  transcribed  onto  the 
computer  by  Word  Processing 
Specialist  trained  to  use  the 
authoring  software 


Editor  edits  text, 
inputs  equations 


Instructional 
Designer  analyzes 
content  for 
relationships,  flow, 
and  organization 


Content  Expert  2  is 
consulted  with 
questions,  works  with 
the  Graphic  Designer 
to  produce  graphics 
and  animations 


Programmer  puts  the 
module  into 
established  interface 


Content  Expert  1  goes 
over  module  with 
Instructional  Designer 
and  explains  content 
using  lots  of  sketches 
and  verbal  explanations 


Content  Expert  1 
goes  over  module  and 
makes  notes  for 
revisions  on  paper 


Instructional 
Designer/  Graphic 
Designer  makes 
revisions 

Editor  checks 
module,  makes 
revisions 

— > 

Content  Expert  1 
checks  module 


i 


Content  Expert  2 
checks  module 


Summary 

For  curriculum  development  projects  in  the  sciences  at  the  postsecondary  level  in  which  the 
instructional  designers  do  not  have  a  science  background,  it  is  important  to  clarify  how  much  faith  the 
content  expert  has  in  the  instructional  designer  in  terms  of  structuring  content,  and  what  their  expectations 
are.  It  is  also  important  for  content  experts  and  instructional  designers  to  speak  the  same  language  with 
regards  to  the  design  of  the  instruction. 


a 

ERIC 


207 


Each  lesson  must  go  through  several  iterations  among  team  members,  whose  roles  should  be 
defined  as  clearly  as  possible.  These  iterations  between  the  team  members  ensures  accuracy  of  content, 
clarity  of  organization,  appropriate  use  of  the  computer's  capabilities,  and  ease  of  navigation.  If  the  content 
expert  does  not  have  the  time  to  complete  several  iterations,  a  secondary  content  expert,  such  as  another 
professor  or  a  graduate  student,  may  be  able  to  bridge  the  gap. 

Other  CBI  Development  Efforts 

Survey 

In  order  to  learn  about  the  design  process  of  other  computer-based  development  efforts  in  the 
sciences,  we  sent  a  survey  to  other  NSF  grant  recipients  who  were  working  on  similar  projects  at  the 
postsecondary  level.  We  also  sent  the  survey  to  an  authoring  discussion  group  on  the  Internet.  We  were 
interested  in  qualitative  information  on  how  they  approach  development,  the  roles  of  team  members, 
whether  or  not  team  members  have  a  science  background,  and,  if  not,  how  they  communicate. 

Results 

The  majority  of  respondents  were  developing  software  to  supplement  a  specific  course,  textbook, 
or  lecture.  They  were  initiated  by  college  faculty  who  were  the  content  experts  for  their  respective  projects. 
Most  of  their  projects  had  been  going  on  from  !  to  4  years.  The  scope  of  the  projects  was  affected  by  the 
size  and  structure  of  the  team.  Projects  that  were  developed  by  individuals  were  narrower  in  scope,  dealing 
with  specific  lesson  segments  to  supplement  class  lectures,  and  generally  consisting  of  practice  problems 
for  the  users.  Team  projects  dealt  with  large  units  of  instruction,  sometimes  at  the  curriculum  level. 

In  order  to  examine  the  design  process  and  the  lines  of  communication  in  team  projects,  we  divided 
the  surveys  into  three  groups:  (1)  content  experts  who  work  mostly  on  their  own,  (2)  content  experts  who 
work  with  a  team  in  which  all  members  have  a  science  background,  and  (3)  content  experts  who  work  with 
a  team  in  which  not  all  members  have  a  science  background. 

Development  Approach 

It  seemed  that  the  division  of  labor  in  a  project  where  all  members  had  a  science  background  did 
not  always  take  advantage  of  specialized  skills.  In  some  projects,  the  entire  module  was  divided  into  lesson 
segments,  and  each  member  would  work  on  a  particular  segment,  doing  all  the  writing,  programming  and 
graphics.  For  teams  in  which  not  all  members  had  a  science  background,  the  development  approach  was 
more  collaborative,  although  there  was  considerable  overlap  of  roles.  In  the  majority  of  projects  where  not 
all  members  had  a  science  background,  the  content  experts  provided  the  initial  storyboards  and  the 
subsequent  development  was  done  by  the  other  team  members,  and  the  content  expert  continued  to  be 
consulted  throughout  the  development. 

Roles 

When  asked  what  roles  were  played  by  the  team  members,  rarely  was  the  term  instructional 
designer  used.  The  most  commonly  stated  roles  were  content  expert,  graphic  designer,  and  programmer. 
Most  people  filled  more  than  one  role,  and  it  seemed  that  in  particular  programmers  did  a  lot  of  the 
instructional  design  work,  such  as  writing  feedback,  storyboarding  content,  etc. 

Communication 

As  expected,  when  all  team  members  had  a  science  background,  communication  was  not  a 
problem.  In  teams  where  not  all  members  have  a  science  background,  we  were  expecting  respondents  to 
cite  communication  as  a  significant  constraint,  but  that  was  not  the  case.  Apparently,  team  members  have 
overcome  communication  barriers  by  using  strategies  that  helped  bridge  the  gap  between  what  the  content 
expert  is  trying  to  communicate  and  what  the  other  team  members  understand.  Most  of  them  state  that  they 
use  a  lot  of  sketching  and  verbal  explanations,  and  back  this  up  with  frequent  checking  and  clarification  of 
material  by  the  content  expert.  In  several  surveys  the  respondents  indicated  that  an  added  benefit  to  this 
approach  was  that,  in  the  process  of  explaining  things  to  the  other  team  members,  the  content  members 
learned  to  be  more  focused,  and  to  make  instruction  more  explicit. 
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Constraints 

The  most  commonly  listed  constraints  were  financial  and  time.  The  majority  of  professors  who 
responded  stated  that  it  was  difficult  to  devote  enough  time  to  the  development  of  computer-based 
instructional  projects  because  of  other  demands,  and  because  professors  may  not  receive  publication  credit 
for  software. 


Many  teams  are  working  together  successfully  to  design  and  develop  science  software,  despite  the 
fact  that  not  all  team  members  have  a  science  background.  Survey  results  indicate  that  the  communication 
gap  is  not  seen  as  a  constraint.  The  use  of  verbal  explanations  and  sketching  helps  to  bridge  the  gap.  In 
addition,  content  experts  indicated  that  discussions  with  other  nonscience  team  members  helped  them  to 
improve  their  teaching  strategies  in  the  classroom.  Results  indicate  that  the  role  of  the  instructional 
designer  in  technical  and  nontechnical  content  areas  is  different,  especially  in  the  Analysis  and  Design 
phases  (see  Table  1  for  a  comparison). 


Table  1.  The  Role  of  the  Instructional  Designer  in  Highly  Technical  vs  Nontechnical 
Content  Areas  (assuming  the  ID  has  no  prior  knowledge  with  the  content) 


Highly  Technical  Content  Areas 

Nontechnical  Content  Areas 

Analysis 

The  content  expert  completes  the  needs 
assessment,  identifies  the  instructional 
problem,  and  does  the  task  analysis. 

The  instructional  designer's  role  is  to  ask 
relevant  Questions  to  guide  the  content 
expert  (for  example,  "What  is  the  gap 
between  what  students  should  know  and 
what  they  actually  know?  What  are  the 
goals,  learner  characteristics?"  etc.). 

The  instructional  designer  completes  the 
needs  assessment,  problem  identification, 
and  does  the  task  analysis. 

Design 

The  content  expert  writes  objectives, 
develops  test  items,  and  plans  the 
instruction. 

The  role  of  the  instructional  designer  is  to 
provide  examples  of  instructional 
strategies  that  the  content  expert  might 
adapt,  and  to  ask  questions  that  clarify  the 
objectives,  the  content  structure,  the  kind 
of  learning  that's  supposed  to  take  place, 
the  kind  of  activities  that  will  best 
facilitate  learning,  the  graphics  that  go 
with  the  lesson,  etc.  (for  example,  "How 
would  you  approach  a  lesson/concept? 
Problem  first  or  description  first?  What  is 
a  i  od  example  of  this  relationship? 
Should  the  lesson  branch  out  at  this  stage? 
If  yes,  where  to?"). 

The  instructional  designer  writes 
objectives,  develops  test  items,  and  plans 
the  instruction. 

The  content  expert  serves  as  consultant, 
and  answers  specific  content-related  and 
structural  questions,  clarifies  terms, 
jargon,  etc. 
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Table  1  (continued).  The  Role  of  the  Instructional  Designer  in  Highly  Technical  vs 
Nontechnical  Content  Areas  (assuming  the  ID  has  no  prior  knowledge  with  the  content) 


Development 

The  content  expert  generates  storyboards 
that  establish  the  sequence  of  instructional 
events. 

The  role  of  the  instructional  designer  is  to 
gain  as  much  familiarity  with  the  content 
as  possible,  to  try  to  recognize 
relationships  in  the  content,  and  to  make 
sure  that  there  is  flow  and  consistency. 
The  instructional  designer  and  the  content 
expert  work  together  to  make  decisions 
about  media. 

The  instructional  designer  writes  the 
content,  develops  storyboards,  develops  a 
prototype,  and  goes  back  to  the  content 
pvnerf  fc\r  fpjvlhark 

Implementa- 
tion 

The  instructional  designer  prepares 
necessary  documentation  and  oversees 
installation. 

The  instructional  designer  prepares 
necessary  documentation  and  oversees 
installation. 

Evaluation 

The  instructional  designer  evaluates  the 
materials  based  on  the  objectives  and  test 
items  generated  by  the  content  expert.  The 
content,  expert  and  instructional  designer 
work  together  to  make  revisions. 

The  instructional  designer  evaluates  the 
materials  and  makes  necessary  revisions. 

The  content  expert  serves  as  a  consultant. 

Existing  Science  Software 

We  were  interested  in  finding  out  whether  or  not  existing  science  software  for  higher  education 
demonstrated  evidence  of  effective  instructional  design:  (1)  orienting  information,  (2)  screen  design,  and  (3) 
navigation,  and  (4)  effective  use  of  the  media.  We  were  not  able  to  evaluate  the  actual  content,  since  the 
software  packages  dealt  with  a  wide  variety  of  highly  technical  content  areas.  The  software  that  we  reviewed 
included  a  sample  of  computer-based  modules  that  are  currently  packaged  with  science  textbooks  at  the 
University  of  Minnesota  bookstore,  and  science  software  packages  that  we  obtained  by  word  of  mouth. 

Results 

All  of  the  software  that  we  looked  at  was  intended  to  be  used  as  a  supplement  to  a  textbook  or 
lecture,  in  varying  degrees.  Some  of  the  software  consisted  of  complete  instructional  modules.  Most  of  the 
software  that  was  packaged  with  a  textbook  simply  provided  practice  problems  for  particular  segments  of  a 
lesson  or  course. 
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Orienting  information 

Many  of  the  applications  that  we  looked  at  consisted  of  a  main  menu  that  listed  different  sections 
that  could  be  accessed  by  clicking  on  them  with  the  mouse.  Across  all  of  the  software  that  we  reviewed, 
there  were  no  goals  or  performance  objectives,  and  very  little  feedback.  The  feedback  generally  consisted  of 
"correct"  or  "incorrect"  with  no  elaboration.  The  format  generally  consisted  of  an  instructional  unit 
followed  by  practice  questions.  The  practice  questions  were  generally  multiple  choice,  and  the  majority  of 
the  questions  required  the  user  to  input  a  correct  answer  before  continuing.  Some  of  the  practice  questions 
asked  the  user  to  input  data,  and  these  also  required  a  correct  number  to  be  input  before  moving  on.  None 
of  the  modules  contained  an  on-line  glossary,  and  only  one  module  had  a  help  button.  None  of  the  modules 
provided  page  numbers,  or  any  means  to  let  the  user  know  how  much  they  had  completed  or  how  much  was 
left. 

Screen  design 

The  software  ranged  from  those  that  used  effective,  simple  graphics  and  animations  to  those  that 
were  purely  text-based.  The  software  that  was  mainly  text-based  appeared  to  be  better  suited  for  paper-based 
presentation.  For  the  software  that  incorporated  graphics  and  animations,  the  font  used  was  easy  to  read, 
but  the  on-screen  text  generally  appeared  to  be  excessive.  In  general,  elements  on  the  screen  (text,  graphics, 
navigational  buttons)  appeared  in  different  locations  on  the  screen  in  different  sections  of  the  lesson. 

Navigation 

The  software  ranged  from  that  which  presented  the  information  in  a  linear  fashion  to  software  that 
allowed  the  user  to  jump  to  different  sections,  or  to  different  segments  entirely,  such  as  problem-solving  or 
experiments.  However,  across  all  of  these  different  approaches,  the  majority  did  not  have  a  means  for  the 
user  to  back  up.  In  the  software  that  provided  a  main  menu,  the  users  always  had  the  option  to  jump  back 
to  the  main  menu,  but  not  to  back  up  within  the  section. 

Effective  use  of  the  media 

Aside  from  the  purely  text-based  approaches,  the  majority  of  software  packages  that  we  reviewed 
took  advantage  of  the  computer's  ability  to  visually  demonstrate  abstract  concepts.  The  graphics  and 
animations  were  simple,  yet  effectively  demonstrated  the  concepts.  Most  of  the  interactivity  of  these 
modules  was  navigational,  where  the  user  selected  the  section  of  the  lesson  that  they  wanted  to  access. 
Some  of  the  modules  had  simple  interactions  in  which  the  user  input  data  and  clicked  a  button  to  see  the 
graphic  result.  A  few  of  the  modules  incorporated  simulations  in  which  the  user  could  manipulate  data  in 
the  context  of  solving  a  larger  problem.  Most  of  the  interactivity  was  not  incorporated  into  the  instruction, 
but  existed  as  a  separate  section. 

Summary 

There  seemed  to  be  a  general  lack  of  instructional  design  components  in  the  software  packages  that 
we  reviewed.  Each  package  had  at  least  one  component  of  effective  instructional  design,  but  no  single 
package  demonstrated  all.  For  example,  one  software  package  had  good  graphics  and  animation  but  didn't 
have  any  feedback,  another  package  had  very  effective  help  buttons  but  didn't  allow  the  user  to  move 
backward.  Several  of  the  applications  were  entirely  text-based  and  would  be  better  suited  for  paper 
presentation.  All  of  the  software  lacked  goals  and  objectives.  Many  of  the  modules  had  practice  questions, 
but  there  was  not  much  feedback  and  it  generally  consisted  of  "incorrect"  and  "correct,"  and  most  questions 
required  a  correct  answer  before  continuing.  Very  few  of  the  modules  we  looked  at  were  interactive,  and 
most  of  the  interactions  consisted  of  the  user  inputting  data  and  pushing  a  button  to  see  a  graphic  result, 
without  showing  the  process. 


Conclusion 


Through  visualization,  simulations,  and  interactions,  the  computer  has  great  potential  for 
improving  science  education  at  the  postsecondary  level.  Many  currently  available  software  packages  on  the 
market  today  do  not  seem  to  take  advantage  of  the  computer's  capabilities,  thus  the  application  of 
instructional  design  principles  to  future' computer-based  development  efforts  in  the  sciences  could  help  to 
produce  more  instructionally  sound  software. 
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It  is  possible  for  instructional  design  theory  to  be  applied  to  higher  education  science  content.  The 
primary  difference  between  technical  and  nontechnical  instructional  design  approaches  is  that  in  technical 
content  areas,  the  instructional  designer's  role  is  to  guide  the  content  expert  through  the  analysis  and  design 
stages,  and  there  are  more  iterations  with  the  content  expert  throughout  the  entire  development  process. 

It  is  not  necessary  for  instructional  designers  to  become  completely  immersed  in  the  content,  but  it 
is  important  for  them  to  become  as  knowledgeable  as  possible  in  order  to  ask  relevant  and  guiding 
questions.  It  is  important  to  develop  common  vocabulary  about  approaches  to  instructional  principles. 
The  communication  process  between  content  experts  and  instructional  designers  can  be  facilitated  through 
the  use  of  verbal  explanations  and  sketches,  and  other  professors  or  graduate  students  may  help  to  bridge  the 
gap  between  the  knowledge  of  content  experts  and  the  backgrounds  of  instructional  designers. 
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Introdution 


The  importance  of  feedback  to  facilitate  learning  has  long  been  acknowledged  (Kulhavy  1977- 
Mory,  1992;Schimmel,  1986;  Bangert-Drowns  et  al,  1991).  Appropriate  feedback  confirms  the  learner's 
expectancy,  directs  attention  to  relevant  factors,  and  stimulates  recall  of  relevant  skills  and  knowledge 
(Gagne,  1985).  Computer-based  instruction  (CBI)  opens  new  avenues  for  increasing  the  variety  of  possible 
feedback  strategies  that  a  teacher  may  employ  to  optimize  learner  performance.  However,  there  is  little 
empirical  research  to  indicate  that  educational  gains  can  be  achieved  by  simply  incorporating  speech  as 
feedback  on  cognitive  aspects  of  psychomotor  domain. 

The  effectiveness  and  impact  of  speech  in  such  roles  as  the  delivery  of  content,  feedback  and 
motivation  in  classroom  instruction  are  well  documented  (Brophy,  1981,  1986;  Sales  &  Johnston,  1993) 
Sanders,  Benbasset  and  Smith  (1976),  for  example,  argue  that  for  speech  to  add  significantly  to  learning,  it 
must  not  "merely  take  the  place  of  a  hard-copy  manual  or  of  printed  text  on  a  computer  terminal."  Saloman 
(1 979,  1985)  also  argues  that  research  on  the  effectiveness  of  media  must  look  at  the  most  essential 
characteristics  of  symbol  systems  being  utilized.  In  software  that  utilizes  digitized  speech,  Brophy  (1986) 
indicates  that  these  essential  characteristics  are  the  familiarity  of  the  source,  the  nature  of  the  learners 
relationship  with  the  source,  and  the  attitude  communicated  through  tones  and  inflections  in  feedback  to  the 
learner. 

Feedback  effectiveness  is  influenced  by  the  nature  of  the  learning  task  and  student  ability.  For 
example,  KCR  was  most  effective  than  AUC  feedback  for  low  ability  students  (Dick  &  Latta  ,  1970)  In  a 
recent  studies,  Clariana  and  Smith  (1989,  1991)  suggested  that  higher  ability  students  benefit  from  the 
additional  information  provided  by  elaborative  feedback.  Furthermore,  low  ability  students  performed 
poorly  with  elaborative  feedback.  Speech  can  be  a  critical  feedback  component  of  instruction.  It  appears  to 
be  particularly  valuable  as  a  means  of  commutation  between  the  teacher  and  the  learner.  To  date,  however, 
very  little  research  has  been  conducted  which  looks  at  the  effects  of  digitized  feedback  under  different  learner 
characteristics  (e.g.,  high-  or  low-prior  knowledge)  on  cognitive  aspects  of  psychomotor  performance. 

The  purpose  of  this  study  was  to  investigated  the  effects  of  digitized  feedback  and  ability  on  the 
achievement  of  college  students  during  computer-based  instruction.  The  achievement  of  high  and  low-prior 
knowledge  students  was  compared  among  different  types  of  feedback  treatments. 


Method 

Subjects 

A  sample  of  68  university  students  from  four  sections  of  beginning  tennis  classes  at  the  Chung 
Cheng  University  participated  in  the  study.  Student's  preexisting  tennis  knowledge  was  tested  to  categorize 
them  into  high  and  low  prior  knowledge  groups.  High  prior  knowledge  students  were  defined  as  those  with 
pretest  scores  above  the  55th  percentile,  while  low  prior  knowledge  students  were  defined  as  those  with 
scores  below  the  45th  percentile.  In  order  to  control  for  the  ability  of  learning  in  the  study,  the  middle  109. 
of  the  students  (those  falling  between  45th  and  55th  percentile)  were  not  included. 

Materials 

CBI  Lgsson  Content.  The  computer-based  instructional  unit  used  in  this  study  were  an  interactive  video 
lesson  on  cognitive  area  of  tennis  skill  performance.  The  interactive  video  instructional  module  employed 
was  designed  to  help  the  beginners  build  up  there  basic  foundation  in  tennis  skills.  By  carefully  selecting 
the  Beginning  of  Tennis  instructional  video  program,  the  lesson  included  four  sections:  a)  The  Grip;  b) 
Correct  swing  technique;  c)  Approach  shots;  and  d)  Terminology  and  Rules.  During  the  lesson,  students 
received  immediate  feedback  in  specific  skill  areas  as  well  as  valuable  experience  in  working  with  questions 
and  problems  posed  in  the  style  of  the  computer-delivered  post  test.  All  versions  of  the  lesson  provided  the 
same  information,  examples,  and  practice  on  the  concepts  of  learning.  The  average  of  three  versions 
required  50  minutes  for  the  subject  to  complete  the  interactive  tutorial  lesson. 

Retention  test.  Two  weeks  later,  subjects  take  a  40-item  test  in  a  paper-and-pencil  format.  In  order  to 
maintain  reliability  and  consistency,  subjects  would  not  be  informed  of  the  retention  test  until  it  was 
administered.  The  Spearman-Brown  formula  was  used  to  calculate  the  split-half  reliability  of  both  question 
sets.  The  reliability  of  the  lesson  immediate  posttest  questions  was  r=0.70. 
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Attitude  Questionnaire.  Following  the  lesson,  students  responded  to  a  Likert-type  questionnaire  developed 
by  author.  This  instrument  contained  ten  items.  It  was  designed  to  collect  data  on  the  usefulness,  quality, 
weaknesses,  and  overall  feeling  that  the  students  had  for  the  interactive  video-delivered  instruction.  Subjects 
responded  to  both  positively  and  negatively  worded  statements  by  marking  their  opinions  on  a  scale  from  1 
(strongly  agree)  to  5  (Strong  disagree).  The  Cronbach's  Coefficient  Alpha  reliability  for  the  questionnaire 
was  .92.  An  typical  item  on  the  questionnaire  was,  "The  program  was  easy  to  use". 

Experimental  Procedures 

Based  on  subjects'  prior  knowledge  test  scores  (preexisting  knowledge  of  tennis),  students  were 
randomly  assigned  to  a  computer  and  completed  one  of  three  treatment  version:  spoken  audio  only-KOR, 
knowledge  of  the  correct  response  -KCR( voice  with  text),  and  elaboration  feedback  -EF(voice  with  text, 
visual  and  animation).  To  resolve  logistical  constraints,  the  study  was  implemented  during  a  four-week 
period.  All  students  received  two  days  of  computer-based  instruction  with  an  interactive  video  on  the  four 
sections  of  beginning  tennis  program.  Each  student  spent  about  45-55  minutes  each  day  to  finish  two 
section  content.  Two  weeks  after  the  above  activities,  all  subjects  were  individually  administered  the 
retention  test(paper-and  pencil).  Seven  absentees  completed  the  pretest  one  week  late.  However,  several 
other  students  could  not  complete  the  posttest  and  retention  test.  The  total  of  fifty-five  subjects  completed 
the  study. 

Design  and  Data  Analysis 

The  study  employed  a  3  X  2  factorial  design.  The  first  factor  was  feedback  with  three  levels: 
spoken  audio  only-KOR,  knowledge  of  correct  response-KCR(voice  with  text)  and  elaborative  feedback- 
EF( voice  with  text,  and  visual)  and  second  was  ability  (High  versus  Low-prior  knowledge).  In  analyzing 
data,  ANOVAs  were  conducted  on  each  set  of  variables.  The  dependent  variables  of  the  study  were 
achievement  on  the  delayed  retention  test  and  attitude  measure  scores.  Calculations  were  :nade  using  the 
Statistical  Package  for  the  Social  Sciences  (SPSS)  4.0  for  the  Macintosh,  (©  1990,  SPSS  Inc.).  All  tests 
of  significance  adopted  an  alpha  level  of  .05,  unless  otherwise  indicated. 


Results 

Achievement 

Achievement  v/as  operational ized  as  an  individual's  score  on  the  retention  posttest.  Means  and 
standard  deviations  of  student  performance  on  retention  test  are  given  in  Table  1.  The  results  of  the 
ANOVA  showed  significant  main  effects  for  Feedback,  E(2,49)  of  6.43,  p=.0O3.  for  Ability,  Ed  .49)  of 
4.70,  p=.035.  High  ability  students  (M  =  3 1.53)  performed  better  than  low  ability  students  (M  =  29.69). 
The  feedback  main  effect  was  further  analyzed  via  Scheffe  follow-up  comparisons  of  the  three  overall 
treatment  means.  Results  indicated  the  elaborative  feedback  (EF)  conditions  (M  =  32.50)  was  higher  than 
the  KOR  and  KCR  feedback  conditions  (M  =  29.28  and  M  =  31 .1 1).  However,  there  were  no  significant 
differences  between  KOR(spoken  only)  and  KCR  feedback(spoken  with  text)  condition.  The  interaction 
between  level  of  ability  and  type  of  feedback  was  not  significant. 
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Table  1. 

Means  and  Standard  Deviations  for  Retention  Test  Scores. 


Combined 


FEEDBACK  . 

KOR  KCR  EF  Total 


Ability 

High               Mean  29.25  33.00  32.20  31.53 

SD  3.41  2.88  2.25  3.15 

N  8  8  10  26 

Low                Mean  29.30  29.11  32.80  29.69 

SD  3.95  2.80  2.59  3.14 

N  10  9  10  29 


Mean  29.28  31.11  32.50  30.61 

SD  3.68  2.84  2.42  3. IS 

N  18  17  20  55 


Attitude  Measure 

Means  and  standard  deviations  for  attitude  scores  are  presented  in  Table  2.  Generally  speaking,  no 
matter  which  ability  levels  and  what  treatment  conditions  the  subjects  were  categorized,  they  rated  the 
instructional  program  closer  to  the  positive  direction.  The  average  rating  based  on  all  of  the  subjects  across 
all  questions  was  3.32  (SD=.53).  The  mean  attitude  score  for  low  ability  students  was  higher  (M  =  3.34) 
than  the  mean  attitude  score  for  high  ability  students  (M  =  3.28).  Similarity,  the  KOR  feedback  condition 
(M  =  3.48)  resulted  in  highest  mean  attitude  score  than  the  other  two  feedback  conditions  (KCR,  M  =  3.34; 
EF,  M  =  3.14,  respectively). 


Table  2. 

Means  and  Standard  Deviations  for  Attitude  Scores. 


KOR 

FEEDBACK 

KCR 

EF 

Total 

Ability 

High 

Mean 

3.43 

3.60 

2.92 

3.28 

SD 

.68 

.57 

.34 

.53 

IN 

8 

8 

10 

26 

Low 

Mean 

3.54 

3.11 

3.36 

3.34 

SD 

.55 

.45 

.34 

.45 

N 

10 

9 

10 

29 

Combined 

Mean 

3.48 

3.34 

3.14 

3.32 

SD 

.59 

.55 

.39 

.53 

N 

18 

17 

20 

55 
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Discussion 


The  purpose  of  this  study  was  to  investigate  the  effect  of  different  types  of  computer-delivered 
feedback  at  different  learner  ability  levels  in  cognitive  areas  of  psychomotor  skill  performance.  Students' 
attitudes  toward  computer-based  instruction  with  an  interactive  video  were  also  assessed.  At  the  end  of  the 
lesson,  the  students  completed  an  attitude  questionnaire  and  a  14-day  retention  test. 

As  suggested  from  the  results,  the  anticipated  interaction  between  ability  and  feedback  was  not 
confirmel  The  study  revealed  that  the  elaborati  ve  feedback  was  most  beneficial  for  cognitive  areas  of 
psychomotor  skill  learning.  One  plausible  explanation  for  the  results  obtained  is  that  the  elaborative 
feedback  facilitated  the  acquisition  of  skill  by  viewing  an  ideal/prototypical  performance  of  interactive 
videodisc-delivered  instruction.  As  Schwier  (1987)  suggested,  the  marrying  of  computer  instruction  and 
television  provides  a  potentially  powerful  training  medium.  Instructional  video  may  also  reinforce  learning 
in  understanding  the  role  of  observation/perception  in  the  acquisition  of  motor  skills. 

The  results  of  the  study  revealed  that  no  matter  which  ability  levels  or  treatment  conditions  the 
subjects  were  categorized,  they  rated  the  instructional  program  closer  to  the  positive  direction.  The 
difference  is  particularly  noticeable  for  low  ability  students  in  the  KOR  feedback  group  which  recorded  the 
highest  attitude  scores  of  the  groups.  On  the  other  hand,  high  ability  students  in  the  elaborative  feedback 
group  reported  the  lowest  attitude  scores.  Perhaps,  low-ability  students  feel  more  comfortable  and  confident 
after  simple  feedback  responses,  while  high-ability  students  received  too  much  information  which  may  have 
interfered  with  learning. 

Feedback  is  an  essential  construct  for  many  theories  of  learning  and  instruction.  Frequent  and 
consistent  use  of  feedback  is  also  strongly  promoted  in  today's  theoretical  development,  instructional 
practice,  and  educational  psychology.  However,  while  the  benefits  of  feedback  in  general  might  be  taken  for 
granted,  uncertainty  still  exists  as  to  how  to  select  and  optimize  uses  of  different  forms  of  feedback 
depending  on  myriad  psychosocial  characteristics  of  students  and  the  learning  situation.  Further  research  is 
needed  to  clarify  this  uncertainty  and  provide  the  learner  with  the  most  effective  and  efficient  type  of 
learning  environment. 

Additional  research  is  needed  to  determine  whether  higher  cognitive  tasks  may  impact  the  learner's 
prior  knowledge  to  process  feedback  effectively.  Higher  order  cognitive  tasks  involve  a  more  complex 
variety  of  skills,  thus  usually  making  learner  more  difficult  to  attain.  The  use  of  simple  feedback  may  not 
be  enough  to  guide  the  student  through  the  complex  components  of  such  a  task.  Therefore,  the  use  of 
extra-instructional  feedback  should  be  examined  further.  "That  is,  containing  additional  information  from 
outside  the  immediate  lesson  environment  "(e.g.,  new  examples,  analogies,  or  new  information  to  clarify 
meaning).  Also,  of  particular  relevance  for  the  current  study,  motor  skill  acquisition,  in  which  changes  in 
bodily  movements  are  required  for  mastery,  is  an  important  domain  as  yet  largely  unexamined  in  the 
literature  on  feedback  in  technology-assisted  instruction. 

Additionally,  emerging  interactive  technologies  are  gradually  taking  an  important  role  in  schools, 
educators  are  still  facing  a  serious  dilemma.  They  are  either  to  find  more  effective  ways  of  using  these 
technologies  in  the  classroom  or  to  limit  the  access  by  large  groups  of  students.  Researchers  should  also 
investigate  the  nature  and  strategies  of  promotive  interaction  within  group  learning. 

Finally,  researchers  should  examine  the  effects  of  varieties  of  feedback  in  varied  new  and  emerging 
technological  instructional  environments.  Environments  are  becoming  increasingly  available  which  offers 
an  increasing  range  of  feedback  types  (for  example  digital  images,  high-speed  animation,  video,  audio,  and 
speech).  Also,  technologies  utilizing  digitized  full-motion  video  and  virtual  reality  will  be  widely  available 
as  instructional  environments  in  the  near  future,  bringing  with  them  opportunities  for  multi-sensory  forms 
of  feedback.  Understanding  how  to  match  learners  with  feedback  appropriate  to  achieve  desired  learning 
outcomes  in  these  sophisticated  instructional  technology  systems  will  certainly  deserve  increasing  attention 
in  the  near  future. 
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«,hinu    £  P    1P   C°nCern  °  'nstruc!,onaI  technologists  is  the  development  of  generalizable  mechanisms 
wh.ch  enhance  the  acquisition  of  specific  learning  outcomes  that  are  compatible  with  both  the  learner  and 
learning  environment.  While  research  concerning  feedback  and  cooperative  learning  are  prevalent  in 
educational  literature,  there  has  been  little  systematic  research  to  support  whether  various  types  of  feedback 
.n  cooperative ^earning  situations  can  affect  student  performance  outcomes  and  attitudes.  Computer-based 
instruction  (CBI)  opens  new  avenues  for  increasing  the  variety  of  possible  feedback  strategies  mat  a  teacher 
may  employ  to  optimize  learner  performance,  but  it  is  not  clear  which  type  of  feedback  is  most  beneficial 
conditio'^ rCnt  charactenstics        Wgh-  or  low-prior  knowledge)  during  computer-based  cooperative 

,n        Wf  T*  tyP?aal fE^bf k gr°Uped  accordi"g t0  their  characteristics  and  on  their  functions 
(Dempsey  &  Sales,  1993).  No  feedback  (NF)  allows  the  learners  to  progress  through  the  instructional 
sequence  without  receiving  any  external  indication  of  the  degree  to  which  they  understand  the  content 
Knowledge  of  correct  response  feedback  (KCR)  includes  a  statement  that  a  response  is  "right"  or  "wrong" 
and  a  comment  identifying  the  correct  response.  Elaborati ve  feedback  consists  of  all  substantive 
information  contained  in  the  feedback  message.  It  appears  that  elaborati  ve  feedback  seems  to  encourage  the 
construction  of  a  richer  network  of  pathways  to  the  desired  information. 

Feedback  effectiveness  is  influenced  by  the  nature  of  the  learning  task  and  student  ability  For 
example,  KCR  was  most  effective  than  AUC  feedback  for  low  ability  students  (Dick  &  Latta  1970)  In  a 
recent  studies,  Clariana  and  Smith  (1989,  1991)  suggested  that  higher  ability  students  benefit  from  the 
additional  information  provided  by  elaborative  feedback.  Furthermore,  low  ability  students  performed 
poorly  with  elaborative  feedback. 

Substantial  evidence  exists  to  suggest  that  cooperative  learning  structures  can  promote  positive 
relationships  and  efforts  to  achieve  that  are  key  to  maximizing  the  potential  of  the  technology  (Hooper  & 
Hannafin,  1988,  1991;  Johnson  &  Johnson,  1989).  Cooperative  learning  also  appears  to  foster  higher  level 
cognitive  reasoning,  increased  achievement  and  retention,  and  higher  level  conceptual  understanding 
(Johnson  &  Johnson,  1989).  Students  who  explain  lesson  material  to  a  partner  appear  to  generate 
elaborative  connections  between  new  and  existing  information,  resulting  in  deeper  processing  of  lesson 
content  (Webb,  1982).  To  date,  however,  very  little  research  has  been  conducted  which  looks  at  the  effects 
of  feedback  under  different  learner  characteristics  (e.g.,  high-  or  low-prior  knowledge)  during  computer-based 
cooperative  environment. 

The  purpose  of  the  study  was  to  examine  possible  interactions  among  different  types  of  computer- 
delivered  feedback  on  cognitive  learning  outcomes  and  attitudes  during  computer-based  cooperative 
condition.  The  ach.evement  of  high  and  low-prior  knowledge  students  was  compared  among  different  types 
of  feedback  treatments. 


Method 

Subjects 

Subjects  were  120  university  students  enrolled  in  six  sections  of  beginning  weight  training  at  the 
University  of  Minnesota.  Students  were  classified  as  high-or  low-prior  knowledge  according  to  performance 
on  the  pretest  of  pre-existing  knowledge  of  weight  training.  The  students  were  randomly  assigned  within 
prior-knowledge  groups  to  one  of  three  types  of  feedback  treatments.  High  prior  knowledge  students  were 
defined  as  those  with  pretest  scores  above  the  55th  percentile,  while  low  prior  knowledge  students  were 
defined  as  those  with  scores  below  the  45th  percentile.  For  the  purpose  of  creating  heterogeneous  pairs  of 
students  for  cooperative  learning  and  as  a  way  of  reducing  the  classification  error,  the  middle  10%  of  the 
students  (those  falling  between  45th  and  55th  percentile)  were  not  included  in  data  analysis. 

Materials 

The  CPI  lesson,  The  computer-based  instructional  module  used  in  the  study  was  designed  to  help 
the  beginners  build  their  basic  cognitive  foundation  in  weight  training.  During  the  lesson,  students  received 
immediate  feedback  in  specific  skill  areas  and  questions  posed  in  the  style  of  the  computer-delivered  practice 
test.  This  unit  is  designed  for  self-instruction  and  group  learning.  The  unit  can  be  integrated  with  the  basic 
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level  of  curriculum  presented  in  typical  weight  training  classes  and  used  in  conjunction  with  already 
existing  materials  and  standard  texts  such  as  Hatfield  and  Krotee'  Personalized  Weight  Training  (1984). 

The  content  presented  in  the  module  consists  of  four  sections:  1 )  Terminology  related  to  resistance 
training.  2)  Safety  in  the  weight  room.  3)  Warm  up  and  cool  down  protocols.  4)  Correct  lifting 
technique.  In  pilot  studies,  the  instructional  lesson  was  reviewed  and  revised  by  two  content  experts  and 
four  weight  training  instructors.  The  instructional  lesson  was  fonnatively  evaluated  using  students  of  the 
target  group,  and  was  revised  into  its  final  form.  In  addition,  data  from  the  pilot  study  indicated  its 
effectiveness  in  helping  students  learn  concepts.  That  is,  students  who  completed  the  module  learned  the 
material  to  a  higher  degree  of  competency  than  did  those  who  did  not  go  through  the  unit. 

Retention  test.  Two  weeks  after  completing  the  computer-based  instruction,  all  students  received  a 
paper-and-pencil  retention  test.  This  multiple-choice  test  contained  40-items ,  based  on  knowledge  and 
application  of  learning  outcomes  to  test  students'  factual  recall  and  conceptual  understanding  of  the  lesson. 
These  items  were  previously  chosen  from  the  pilot  study  involving  the  same  content  and  a  similar 
population.  Cronbach's  alpha  method  was  used  to  calculate  the  reliability  of  the  retention  test,  and  was 
found  to  have  a  value  of  .82.  The  mean  of  the  40  item  retention  test  was  28.2  with  a  standard  deviation  of 
4.50. 

Attitude  questionnaire.  Following  the  lesson,  students  responded  to  a  Likert-type  questionnaire 
developed  by  Simsek  &  Tsai  (1992).  This  instrument  was  designed  to  measure  students'  reactions  to 
cooperative  learning.  It  included  27  items  containing  a  5  point  scale  ranging  from  "Strongly  Agree"  to  " 
Strongly  Disagree".  The  items  were  equally  divided  among  three  categories:  attitudes  toward  delivery 
system,  attitudes  toward  subject  matter,  and  attitudes  toward  group  work. 

The  Coefficient  alpha  reliability  for  the  questionnaire  was  reported  to  be  ,91  (Simsek  &  Tsai, 
1992).  Although  those  authors  used  the  questionnaire  with  sixth  grade  students,  the  current  study  still 
found  a  high  reliability  (.86)  for  the  instrument,  even  though  the  subjects  here  were  adults.  The  purpose  of 
the  scale  assessing  attitudes  of  students  was  to  determine  students'  liking  to  the  delivery  system,  the  subject 
matter,  and  of  working  with  a  partner.  An  example  item  for  each  category  was:  "I  enjoyed  working  with 
the  computer"  (delivery  system);  "I  would  like  to  learn  more  about  v/eight  training"  (subject  matter);  and  "I 
feel  more  comfortable  working  in  a  small  group  than  working  alone"  (group  work). 

Procedures 

The  study  was  implemented  during  a  six-week  period.  The  pretest  was  administered  during  the  first 
week.  The  basic  cooperative  skills  training  was  completed  by  all  students  and  occurred  during  the  second 
week.  During  the  third  week,  each  student  completed  two  computer-based  instructional  units.  Each  student 
spent  about  40  to  50  minutes  each  day  to  finish  two  units  of  content.  During  the  fourth  and  fifth  weeks, 
nothing  related  to  the  study  happened  for  the  students.  The  retention  test  was  delivered  in  the  sixth  week  of 
the  study. 

Subjects  were  assigned  to  treatments  using  stratified-random  sampling.  Initially,  high-  and  low- 
prior  knowledge  students  were  randomly  assigned  to  paired.  Next,  within  each  class,  subjects  in  the  paired 
treatment  were  ranked  within  each  ability  group.  Partners  were  assigned  by  combining  students  with 
identical  ranks.  The  most  able  high-  and  low-prior  knowledge  students  were  paired,  as  were  the  second  most 
able  and  so  on.  Thus,  equivalent  heterogeneity  among  group  members  was  established.  Each  group 
contained  two  subjects. 

The  study  was  conducted  in  the  weight  room  of  the  kinesiology  department.  Sixteen  Macintosh 
SE  computers  were  set  up  before  each  class.  Specific  directions  for  implementing  individual  versus 
cooperative  strategies  were  provided  for  each  condition.  Subjects  were  assigned  to  a  computer  and 
completed  one  of  three  types  of  feedback  version  of  the  CBI  lesson.  Upon  completion  of  the  lesson,  each 
subject  individually  completed  the  attitude  questionnaire.  Two  weeks  after  the  above  activities,  all  subjects 
were  individually  administered  the  delayed  retention  test. 

The  basic  cooperative  skills  training  was  adapted  from  procedures  recommended  by  Johnson  and 
Johnson  (1984).  Students  were  presented  with  a  50  minute  instructional  module  development  by  the 
researcher  for  use  with  the  four  weight  training  instructors  intended  to  guide  students  toward  better  use  of 
cooperative  behaviors.  Two  activities  were  given  to  the  cooperative  learning  groups'  instructors  to  help 
reinforce  those  behaviors  students  need  in  order  to  work  cooperatively. 
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Design  and  Data  Analysis 

The  study  employed  a  3  x  2  factorial  design.  The  first  factor  was  Types  of  feedback  with  three 
levels  (no  feedback-NF,  knowledge  of  correct  response-KCR,  and  elaborative  feedback-EF),  and  the  second 
was  Ability  had  two  levels  (high  and  low-prior  knowledge).  In  analyzing  data,  two-way  ANOVA  was  used. 
Dependent  variables  of  the  study  were  achievement  on  the  retention  posttest,  and  attitudes  toward  computer- 
based  instruction,  weight  training,  and  learning  in  pairs.  The  alpha  level  was  set  at  .05,  unless  otherwise 
indicated. 


Results 

Achievement 

Achievement  was  operationalized  as  an  individual's  score  on  the  retention  posttest.  Mean  scores 
and  standard  deviations  for  retention  posttest  scores  can  be  found  in  Table  1.  Results  of  the  ANOVA 
showed  significant  main  effects  for  feedback  type  F(2,  1 14)  =  5.95,  p  <  .31,  and  for  ability  F(l,  114).= 
17.57,  p  <  .001.  A  two-way  interaction  was  found  between  Feedback  and  Ability  F(l,  1 14).=  13.99,  p  < 
.001.  The  feedback  main  effect  was  further  analyzed  via  Scheffe  follow-up  comparisons  of  the  three  overall 
treatment  means.  Results  indicated  the  EF  mean  (M=  33.00)  and  the  KCR  mean  (M=  31 .63)  were  both 
significantly  higher  than  NF  mean  (M=  27.00).  However,  there  were  no  significant  differences  between 
KCR  and  EF  conditions.  A  simple  inspection  of  the  means  shows  students  in  the  high-prior  knowledge 
treatment  (M=  31.95)  outperforming  those  in  the  low-prior  knowledge  treatment  (M=  29.13). 

The  significant  interaction  between  Feedback  and  Ability  was  further  analyzed  by  comparing  the 
six  feedback  means,  using  a  Scheffe's  procedure  to  analyze  the  15  possible  pair-wise  comparisons.  The 
results  of  this  analysis  are  shown  in  Table  3.  Findings,  although  not  significant,  indicated  that  for  low- 
prior  knowledge  students,  the  KCR  condition  mean  was  higher  than  that  for  the  elaborative  feedback  (EF) 
condition.  However,  for  high-prior  knowledge  students,  the  elaborative  feedback  (EF)  condition  mean  was  a 
significantly  higher  than  the  KCR  condition  mean.  This  proved  to  be  significant  level.  That  is,  high-prior 
knowledge  students  performed  better  under  the  EF  feedback  condition.  However,  the  assumption  that  KCR 
feedback  condition  is  better  than  EF  feedback  condition  for  low-prior  knowledge  students  was  not  confirmed. 
The  interaction  is  graphically  displayed  in  Figure  1. 
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Figure  1. 

Graph  of  the  Interaction  of  Feedback  and  Ability  (prior  knowledge)  by  Retention  Posttest. 
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Table  1. 

Means  and  Standard  Deviations  for  Retention  posttest  Scores. 


FEEDBACK 


NF 

KCR 

EF 

Total 

T  T*  L 

High 

M 

28.  /3 

J  1 .33 

IS  SS 

JJ.JJ 

SD 

2.05 

1.86 

1.79 

3.43 

n 

Oft 

zu 

on 
zu 

zu 

ATI 

Low 

M 

25.25 

31.70 

30.45 

29.13 

SD 

3.09 

2.66 

1.77 

3.84 

n 

20 

20 

20 

60 

Combined 

M 

27.00 

31.63 

33.00 

30.54 

SD 

2.57 

2.26 

1.78 

3.64 

n 

40 

40 

40 

120 

Attitudes 

Student  attitudes  toward  the  delivery  system,  subject  matter,  and  group  work  were  measured  with 
the  attitude  questionnaire.  Attitudinal  data  for  108  out  of  the  original  120  subjects  were  included  in  the 
analyses  of  the  results  because  scores  on  one  or  more  measures  were  unavailable  for  the  remaining  12 
subjects  (For  random  reasons:  illness,  transportation  problem,  etc.).  Means  and  standard  deviations  for 
overall  attitude  scores  are  contained  in  Table  2. 


Table  2. 

Means  and  Standard  Deviations  for  Attitude  Scores. 


NF 

FEEDBACK 
KCR 

EF 

Total 

Cooperative 

High 

M 

113.73 

111.75 

111.50 

112.33 

SD 

2.05 

2.90 

4.21 

3.37 

n 

18 

17 

18 

53 

Low 

M 

116.00 

114.06 

114.15 

114.70 

SD 

2.59 

4.07 

2.14 

3.06 

n 

17 

18 

19 

54 

Combined 

M 

1 14.87 

112.91 

112.83 

113.52 

SD 

2.32 

3.49 

3.18 

3.22 

n 

35 

35 

37 

107 

2 -j  i 
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Two-way  ANOVA  results  revealed  that  the  main  effects  were  significant  for  Feedback,  F(2,95)= 
4.52,  p=.013,  for  Ability,  F(l,95)=  14.92,  p<.001.  However,  no  significant  two-way  interactions  were 
found  between  Feedback  and  Ability.  The  feedback  main  effect  was  further  analyzed  via  Scheffe  follow-up 
pair-wise  comparisons  of  the  three.overall  treatment  means  (see  Table  3).  Results  indicated  no  significant 
pairwise  differences  among  the  NF,  KCR,  and  EF  feedback  conditions.  The  significant  main  effect  was 
probably  due  to  the  pooling  of  the  nearly-identical  means  of  the  KCR  treatment  (M  =  1 12.91)  and  the  EF 
treatment  (M  =  1 12.83)  being  substantially  lower  than  the  NF  condition  (M  =  1 14.87). 


Table  3. 

Mean  Differences  for  Type  of  Feedback  by  High  and  Low  Ability  (prior  knowledge)  Level. 


NF/H 

KCR/H 

EF/H 

NF/L 

KCR/L 

NF/H(28.75) 

• 

KCR/H(31.55) 

2.80 

• 

EF/H(35.55) 

6.80* 

4.00* 

• 

NF/L(25.55) 

3.50 

6.30* 

10.30* 

• 

KCR/L(31.70) 

2.95 

0.15 

3.85*- 

6.45* 

• 

EF/L(30.45) 

1.70 

1.10 

5.10* 

5.20* 

1.25 

*p  <  .05  (Scheffe) 


An  examination  of  the  Ability  main  effect  indicated  that  low-prior  knowledge  students  (M= 
1 14.85)  demonstrated  better  attitudes  than  high-prior  knowledge  students  (M=  1 12.13).  Analyses  of 
variance  on  both  the  attitude  toward  subject  matter  and  the  attitude  toward  group  work  subscales  indicated  no 
significant  main  or  interaction  effects. 

Discussion 

This  study  was  to  investigate  the  effects  of  type  feedback  and  ability  (high-  or  low-prior 
knowledge)  in  a  cooperative-based  instructional  program.  Upon  completion  of  the  lesson,  students 
responded  to  a  retention  posttest  and  an  attitude  questionnaire. 

The  result  of  the  study  yielded  a  significant  interaction  between  types  of  feedback  and  prior 
knowledge  level:  elaborative  feedback  (EF)  was  more  effective  with  high-prior  knowledge  students.  High 
prior  knowledge  students  who  received  elaborative  feedback  demonstrated  significantly  higher  in 
achievement  than  those  who  received  KCR  and  NF  treatment.  Specifically,  high-prior  knowledge  students 
seem  more  able  to  take  advantage  of  the  extra  mental  effort  required  by  elaborative  feedback.  However, 
asking  students  to  provide  responses  on  the  computer  without  activating  greater  mental  effort  (as  in  KCR  or 
NF)  might  frustrate  high-prior  knowledge  learners  and  result  in  lower  performance.  For  low-prior 
knowledge  students,  EF  and  KCR  treatments  significantly  outperformed  the  NF  condition.  However,  there 
were  no  significant  differences  between  KCR  feedback  and  elaborative  feedback  for  low-prior  knowledge 
students,  although  the  KCR  mean  scores  were  slightly  higher  than  elaborative  feedback  (EF)  condition. 

The  significant  interaction  of  feedback  type  by  prior  knowledge  in  this  study  partially  replicates  the 
results  of  Dick  &  Latta  (1970)  and  more  recent  studies  by  Clariana  (1990),  Clariana  &  Smith  (1989),  and 
Smith  (1988)  that  high-prior  knowledge  learners  benefit  most  from  feedback  that  provides  additional 
information,  like  elaborative  feedback,  but  not  from  feedback  that  short-circuits  additional  processing,  such 
as  KCR  feedback.  Low-prior  knowledge  learners,  on  the  other  hand,  should  benefit  most  from  feedback  that 
provides  less  information  than  Uie  elaborative  feedback  condition,  like  KCR  feedback,  and  may  be 
overwhelmed  by  feedback  which  requires  additional  processing,  like  elaborative  feedback.  In  the  current 
study,  however,  although  KCR  slightly  outperformed  the  EF  condition,  there  was  no  significant  difference. 
This  could  be  due  fo  the  nature  of  the  feedback  provided  in  the  lesson:  it  was  highly  graphical,  whereas, 
earlier  studies  employed  textual  feedback.  It  might  be  that  the  amount  of  information  contained  in  pictorial 
feedback  and  perceived  by  low-prior  knowledge  students  is  more  difficult  to  specify. 

Results  on  the  attitude  measure  revealed  that  low-prior  knowledge  students  had  more  positive 
attitudes  than  high-prior  knowledge  students.  One  plausible  explanation  for  the  results  obtained  is  that  the 
low-prior  knowledge  students  in  heterogeneous  groups  feel  more  supported  and  satisfied  than  other  students. 
They  may  also  feel  privileged  because  their  high-prior  knowledge  partners  are  always  available  to  help 
them.  According  to  Deutsch  (1949),  interaction  improves  interpersonal  attraction  when  the  interaction 


helps  individuals  to  achieve  personal  goals.  Johnson  &  Johnson  (1989)  also  indicated  that  for  the  college 
and  adult  studies,  cooperative  experiences  resulted  in  greater  interpersonal  attraction  than  did  competitive  or 
individualistic  experiences  (effect  sizes  =  0.83  and  0.40  respectively).  In  cooperative  learning  groups, 
students'  goals  are  linked.  Thus,  partners  seek  outcomes  that  are  beneficial  to  everyone  in  their  group. 
Such  interdependence  motivates  partners  to  invest  effort  toward  a  mutual  goal,  which  in  turn  helps  foster  a 
caring  and  supportive  environment. 

By  understanding  the  conditions  under  which  feedback  is  most  effective  during  learning,  we  can 
improve  both  our  instructional  theories  and  our  practices.  Feedback  serves  to  provide  conditions  that  result 
in  mutual  influence  between  learners  and  their  environments.  For  researchers  and  developers  in  the  field  of 
educational  technology,  the  implications  of  effective  and  efficient  interactions  between  the  learner  and  the 
learning  task  are  critical,  since  newer  technologies  are  able  to  make  optimum  use  of  such  highly  interactive 
environments.  However,  while  a  great  deal  of  research  has  been  conducted  on  the  feedback  and  cooperative 
learning,  the  effects  of  types  of  feedback  in  group  learning  have  not  been  adequately  investigated  between 
these  instructional  variables.  Based  upon  the  results  of  this  and  other  relevant  studies,  the  following 
recommendations  are  proposed  for  future  research. 

Additional  research  is  needed  to  determine  whether  motivational  factors  may  impact  the  learner's 
prior  knowledge  to  process  feedback  effectively.  For  example,  the  learner's  motivational  states,  attitudes 
about  the  instructional  or  topic,  general  self-perceptions,  learning  styles,  and  self-esteem  levels  may  all  play 
a  part  in  their  prior  knowledge  to  successfully  interact  with  feedback.  A  learner  with  a  poor  self-concept  or 
general  lack  of  self-confidence  is  likely  to  underestimate  his  or  her  prior  knowledge  in  a  majority  of 
situations. 

Researchers  should  examine  other  factors  that  influence  cooperative  learning.  One  such  factor  is 
ability-group  composition.  The  present  study  investigated  cooperative  groups  comprised  of  heterogeneous 
prior  knowledge  pairs  and  compared  their  performance  to  similar  students  working  individually.  Future 
research  might  wish  to  heterogeneous  prior  knowledge  groups  with  homogeneous  groups. 

More  research  should  be  conducted  to  examine  the  relationship  among  the  effects  of  various 
computer-delivered  feedback  and  group  size.  Although  interpersonal  interactions  generate  more  powerful 
feedback  in  cooperative  groups,  studies  do  not  provide  a  clear  answer  regarding  the  ideal  group  size.  In  fact, 
the  number  of  students  in  a  group  may  depend  largely  on  the  tasks  students  involve.  Researchers  may  wish 
to  explore  and  clarify  the  relationship  between  group  size  and  type  of  computer-delivered  feedback  to 
determine  which  can  be  employed  most  effectively  and  efficiently  to  boost  students'  performance. 

Finally,  researchers  should  examine  the  effects  of  varieties  of  feedback  in  varied  new  and  emerging 
technological  instructional  environments.  Environments  are  becoming  increasingly  available  which  offers 
an  increasing  range  of  feedback  types  (for  example  digital  images,  high-speed  animation,  video,  audio,  and 
speech).  Also,  technologies  utilizing  digitized  full-motion  video  and  virtual  reality  will  be  widely  available 
as  instructional  environments  in  the  near  future,  bringing  with  them  opportunities  for  multi-sensory  forms 
of  feedback.  Understanding  how  to  match  learners  with  feedback  appropriate  to  achieve  desired  learning 
outcomes  in  these  sophisticated  instructional  technology  systems  will  certainly  deserve  increasing  attention 
in  the  near  future. 
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This  presentation  is  based  on  two  of  my  essays  that  are  currently  in  press.  The  first,  "Providing  Alternative 
Views  of  Contexts,  Instruction  and  Learning  in  Graduate  and  Continuing  Professional  Education  Courses  in 
Instructional  Development,"  1  presents  primarily  a  critical  inquiry,  qualitative,  action  research  framework  for 
exploring  instructional  development  with  teachers,  developers  and  other  professionals  who  may  work  in  a 
variety  of  disciplines.  The  second,  "How  is  Instructional  Development  a  Social  Practice?  Instructional 
Development  in  a  Postmodern  World,"2  provides  several  discussions  on  the  theory  and  practice  of 
instructional  development  from  a  postmodern  stance.  Underlying  both  of  these  essays  and  their  ideas  is  a 
desire  for  more  discussion  and  activity  regarding  the  social  responsibility  of  our  field.  Both  address  the 
question,  "How  is  instructional  development  a  social  practice?" 

Instructional  development  is  about  people.  My  work  currently,  and  in  the  past,  focuses  on  people  and  their 
life  worlds.  I  am  literally  embedded  in  the  lives  of  others.  I  experience  tension  and  desire  for  movement,  I 
sense  ongoing  contradictions,  and  have  observed  a  sincere  interest  in  learning  about  education  and  its  impact 
on  people,  environments,  and  social  relationships. 

I  am  concerned  that  I  cannot  respond,  responsibly  or  practically,  to  the  question  of  instructional 
development  as  a  social  practice  using  the  traditional  language,  models,  and  ideas  of  educational  technology, 
and  more  specifically,  instructional  development.  Instructional  development  continues  to  promote  models 
of  development  whereby  a  group  of  well  informed  developers  produce  instruction  for  courses  and  programs 
external  to  the  people  they  are  intended  for.  Education  (more  broadly),  despite  attempts  to  encourage 
community  intervention  and  participation,  and  to  promote  alternative  philosophies  and  programs,  is  still 
confined  to  schools,  institutions,  and  organizations  in  which  educational  missions  are  largely  defined  by 
external  forces,  including  economic  and  political  ones. 

This  should  no  longer  be  the  case.  The  very  elements  (discussed  and  argued  in  a  multitude  of  media, 
journals,  and  government  reports)  that  are  impacting  society  and  require  educational  reform,  are  the  same 
elements  challenging  our  profession: 

instability  and  uncertainty  of  the  future 

information  technology  movement 

sociocultural  and  demographic  changes 

lifestyle  changes  (including  work,  school  and  recreation) 

These,  apart  from  information  technology,  are  not  new  challenges.  If  v,e  stop  and  deconstruct  the  present  - 
we  see  a  part  of  our  past.  This  is  why  alternative  perspectives  and  discussions,  not  just  programs  and 
activities,  are  our  responsibility.  When  our  past  is  present,  shouldn't  we  be  responsible  and  critique  that 
past?  Shouldn't  we  begin  to  ask,  "Who  are  we  now?  Does  our  past  help  or  hinder  others?"  More 
importantly,  "Who  are  we  responsible  for?" 

As  a  professional,  I  am  compelled  to  be  responsive  to  the  life  worlds  of  others,  as  well  as  my  own.  I 
suggest  that  it  is  an  imperative  that  we  begin  to  conceptualize  our  social  practice  in  the  following  way: 

Instructional  development  activity  is  located  in  a  gray  area  and  should  conceive  of  itself  as  a  practice  that  is 
not  dedicated  to  solutions,  but  as  a  practice  that  contributes  responsibly  to  the  construction  of  people's 
educational  life  worlds;  their  cultural  as  well  as  technical  meanings 

Having  accepted  this  proposition  as  the  starting  point  for  my  own  reflection,  on  the  following  page  I  offer 
examples  of  my  recent  work  as  an  educator. 
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Life  world 


Social  Practices 


Program 


Residency  Program 


Medical  School  undergoing 
reform:  tension,  resistance, 
fragmentation. 


Interviewing,  listening, 
negotiating,  mobilizing  people, 
educating  people,  developing 
communicative  relationships 
through  people  and  media, 
encouraging  sharing  and 
creativity. 


State  Board  of  Health 


State  requiring  more  rural  Encouraging  collaboration, 

physicians:  fragmentation,  lack  interviewing,  integrating  people 
of  information,  politics.  and  activities,  writing  a  report 


based  on  actual  people  and  events. 


Dental  School 


Dental  School  preparing  for 
reform:  resistance,  past 


communication,  leadership 
issues,  lack  of  information. 


experiences  (negative),  lack  of 


Listening,  interviewing, 
reviewing  information  and 
programs,  visiting, 
providing  information  and 
support,  encouraging. 


Primary  Outcomes  of  a  Critical/Postmodern  Approach:  Greater  social  interaction,  improved 
student  and  faculty  well-being,  movement  and  integration  across  disciplines,  formulation  of  ideas  for  action 
research  and  innovative  projects,  improved  feedback  and  communication,  desire  for  social  and  professional 
development,  increased  motivation  and  commitment  to  educational  activity,  empowerment  of  educational 
community  to  take  responsibility  and  ownership  of  educational  issues,  improved  environment  and 
leadership,  thoughtful  and  dynamic  educational  programs. 


ENDNOTES 

1  Providing  Alternative  Views  of  Contexts.  Instruction  and  Learning  in  Graduate  and  Continuing 
Professional  Education  Courses  in  Instructional  Development  (Jamison.  1994).  First  presented  at  the 
"Center  for  Urban  Ethnography  Conference,"  Philadelphia,  Pennsylvania;  revised  as  a  chapter  for  a  work  in 
progress  (no  book  title  currently  available)  on  developing  courses  and  programs  in  instructional 
development  from  critical  perspectives.  For  more  information  on  this  book  contact  the  editors:  Al 
Januszewski,  Pottsdam  College,  Pottsdam,  New  York  and  Rhonda  Robinson,  Northern  Illinois  State 
University. 

2  How  is  Instructional  Development  a  Social  Practice?  Instructional  Development  in  a  Postmodern  World 
(Jamison,  1995).  Chapter  for  a  work  in  press  (no  book  title  currently  available)  on  current  theory  and 
practice  in  educational  technology.  Charles  Dills,  Editor.  Proposed  publication  date  is  Fall  1995.  For 
further  information  contact:  Larry  Lipsitz,  Educational  Technology  Publications,  Englewood  Cliffs,  New 
Jersey. 
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Instructional  Development:  Traditional  and  Alternative  Frameworks 
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CONTEXTS 


PoSiflcal  Context: 
Language  ami  Power 
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Social  System* 
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Environments!  Context: 
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Development  Context: 
Imagination,  intuition.  Art  and 

Science 


DIFFERENT  WAYS  or  INQUIRINC 
INTO  CONTEXTS 

-Critical  Inquiry 
-Autobiography 

-Ethnography 
-Holistic  Ethnography 
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-Participatory  Action  Research 

-Coitiiiive  Anthropology 
-PhcnoMesiology 
-Sesnlotic  Approach 
-Case- Study 
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-Policy  Research 
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Tff  lip^fll  Model 


Plan  a  framework  of  analysis  to 
identify  the  instructional 
problcm(s) 


Design  a  product  or 
program  to  resolve 
the  problem(s) 

j  Create  the  initial 

|  product  or 
I  program 

Implement  the  product  j 
or  program  1 

Test  and  evaluate  the 

Design  and  create  a 

product  or  program  for 

revised  product  or 

resolution  of  the 

program 

problem 

.J 


Associated  Concepts 

nomothetic  (universal  law,  one  paradigm) 
reflective  of  reality 

order  through  differentiation  and  separation 
passive  and  static 
strives  for  control  and  homeostasis 
surrenders  people  and  settings  to  "the  problem" 
monologic  discourse 
etic  view  (outsider's  perspective) 
imposes  resolution  from  the  outside 
purported  to  be  based  on  the  notion  of 
"naturally  occurring  systems" 


model  dictates  practice 
hierarchical  and  authoritative 
mechanistic 
autonomous 

based  on  supposed  "objective"  reality 
products  and  programs  are  universal  and 
generalizable  to  other  situations 
search  for  function  and  example 


Figure  1  Technical  Model  of  Instructional  Development  and  Associated 
Concepts 
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Lifeworlds 


Associated  Concepts 

idiographic  (case-based) 

search  for  meaning 

embraces  uncertainty  and  ambiguity 

interpretive  and  responsive 

complementary  and  contradictory 

acknowledges  tension  and  struggle  in  settings 

people,  setting  and  developer  are  active 

participants  in  construction  of  lifeworld 
dialogical  and  conversational 
personal  and  metaphorical 
emic  (insider's  perspective) 
bridges  theory  and  practice 
multidimensional 


participatory 

explores  cultural,  subjective  reality 
representative  of  part  of  reality  (partial) 
competing  paradigms  are  viewed  as 

representative  of  complexity 
process  focuses  on  construction  and 

interpretation  of  meaning 
conceptual  and  multiple  meanings  possible 
local  meanings  provide  understanding 
engages  through  exploration 
inquires  into  the  meaning  of  difference 


Figure  2  Critical  Approach  to  Instructional  Development  and  Associated 
Concepts 
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There  are  three  basic  aspects  of  graduate  programs  in  educational  technology.  The  first  focuses  on  the 
essential  concepts,  skills,  principles,  and  procedures  that  students  should  know/or  be  able  to  do  in  order  to 
function  well  in  our  field.  Examples  of  this  sort  of  thing  include  writing  objectives,  doing  task  analysis, 
and  producing  "good"  overhead  transparencies. 

The  second  deals  with  the  theoretical  and  conceptual  bases  in  which  the  essential  skills,  principles,  and 
procedures  are  grounded.  Examples  of  this  include  perception  theory  in  message  design,  S-R  association  in 
behavioral  approaches  to  instructional  design,  and  expertise  oriented  or  connoisseurial  approaches  to 
program  evaluation. 

The  third  is  concerned  with  the  foundations  of  the  field.  Here  foundations  does  not  means  the  introductory 
issues  and  ideas  of  the  field.  Here  it  means  the  "cultural  foundations"  of  the  field.  It  refers  to  those  issues 
and  ideas  which  are  the  foundation  of  the  "culture  of  educational  technology".  Examples  of  this  include  the 
history  and  definition  of  the  field,  ethical  and  professional  issues  of  the  field,  and  the  hidden  implications  of 
our  research  methodologies  and  design  practices.  Most  teaching  and  course  work  in  the  field  had  done  an 
admirable  job  of  dealing  with  the  first  two  of  these  aspects  but  seems  somewhat  reluctant  to  address  the 
third. 

Purpose 

The  purpose  of  this  paper  is  to  describe  the  way  in  which  foundations  issues  are  covered  at  an  institution 
which  grants  masters  degrees  in  education  in  the  area  of  educational  technology. 

There  are  two  ways  in  which  this  is  done  in  the  program  in  educational  technology  at  SUNY  Potsdam.  The 
first  is  to  build  it  into  the  skills  focused  courses.  In  the  course  on  instructional  design  the  is  time  given  to 
the  subject  of  unintended  learning  outcomes.  In  the  course  on  program  evaluation  there  is  time  given  to  the 
different  approaches  and  their  implications  in  program  evaluation.  These  subjects  are  not  a  prime  area  of 
study  in  those  two  courses.  The  intent  behind  including  these  topics  is  twofold:  1)  to  raise  awareness  about 
the  implications  of  our  activities  in  the  field  of  educational  technology;  and  2)  to  raise  the  idea  that  the 
practice  of  the  field  often  covers  much  more  then  is  intended  (we  are  responsible  for  more  than  what  we 
intend). 

The  second  way  in  which  foundations  issues  are  covered  in  the  educational  technology  program  at  SUNY 
Potsdam  are  in  courses  which  were  specifically  designed  to  do  so.  I  teach  two  courses  (one  of  which  is 
required  of  all  students  in  our  program  [their  choice])  (GRED  615)  Critical  Issues  in  the  use  of  Technology 
in  Education  and  (GRED  625)  The  History  and  Philosophy  of  Instructional  Technology  which  focus  on 
foundations  issues.  Both  courses  cover  definitions  of  educational  technology,  differing  conceptions  of 
educational  technology,  and  the  concept  of  professionalism.  Both  course  explore  the  importance  of 
conceptual  clarity.  But  the  similarity  ends  there. 


The  History  and  Philosophy  of  Instructional  Technology    (GRED  625) 


The  purpose  of  this  course  is  to  contribute  to  a  conceptual  understanding  of  educational  technology.  This  is 
done  by  tracing  the  development  of  some  of  the  popular  concepts  and  ideas  in  the  field  of  educational 
technology.  It  includes  identifying  and  analyzing  the  concepts,  root  ideas,  and  strands  of  thought  that  have 
existed  in  educational  technology  since  the  first  official  definition  was  published  in  1963.  This  course  also 
analyzes  the  contributions  of  some  of  the  important  people  in  the  field  of  educational  technology  in  the 
last  fifty  years. 

This  course  should  help  to  answer  the  questions  'why  do  educational  technologists  think  the  way  they  do 
about  educational  technology?'  and  'how  has  that  way  of  thinking  changed  with  time?'.  Historian  George 
Boas  stated,  "the  history  of  ideas  tells  us  among  other  things  how  we  got  to  think  the  way  we  do-and  if  that 
is  not  of  importance  one  wonders  what  is"  (Boas,  1969,  p.  3). 
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In  the  words  of  Ludwig  Fleck: 


Whether  we  like  it  or  not  we  can  never  sever  our  links  with  the  past  complete  with  all  its 
errors.  It  survives  in  accepted  concepts,  in  the  presentation  of  problems  in  everyday  life, 
as  well  as  the  language  and  institutions  that  we  employ.  Concepts  are  not  spontaneously 
created  or  generated  but  are  determined  by  prior  thought  -  (Ludwik  Fleck,  1979,  p.  9). 

Why  history  and  philosophy? 

Actually  this  course  is  about  the  intellectual  history  of  the  field  of  educational  technology.  As  such  it  is 
about  both,  the  history  and  the  philosophy  involved  with  educational  technology,  but  I  tend  to  approach  it 
more  from  a  historical  perspective. 

I  think  that  history  is  strongly  connected  with  the  concept  of  change  in  two  ways.  First,  the  study  of 
history  helps  to  show  how  things  have  changed  and  helps  to  explain  why  they  are  the  way  they  are  at 
present.  Second,  history  provides  the  understanding  which  is  an  essential  component  of  reflective  thought. 
The  study  of  history  can  help  maintain  traditions,  help  individuals  to  stick  to  their  roots,  if  the  roots  are 
valued  strongly  enough.  But  historical  analysis  may  also  provide  more  options.  An  historical 
understanding  can  help  us  to  'break  out'  of  past  patterns  or  shift  emphases  if  it  seems  important  to  do  so. 
History  can  greatly  contribute  to  the  conscious  decisions  that  are  made  about  change. 

Fleck  described  the  problem  of  writing  a  history  of  an  academic  field: 

It  is  very  difficult,  if  not  impossible,  to  give  an  accurate  historical  account  of  an  academic 
discipline  or  a  field  of  study.  Many  developing  strands  of  thought  intersect  and  interact 
with  one  another.  All  of  these  would  have  to  be  represented;  first,  as  continuous  lines  of 
development  and,  second,  in  everyone  of  their  many  intersections  and  connections.  Third, 
the  main  direction  of  the  development,  taken  as  an  'idealized  average',  would  have  to  be 
described  separately  and  at  the  same  time.  The  continuity  of  the  line  of  thought  that  has 
already  been  mapped  out  must  continually  be  interrupted  to  introduce  other  lines  of 
thought.  The  main  current  of  thought  would  often  have  to  be  held  up  in  order  to 
investigate  and  explain  any  connections.  Often,  much  has  to  be  omitted  to  preserve  the 
main  current.  Instead  of  a  description  of  dynamic  interactions  one  is  often  left  with  an 
artificial  and  arbitrary  scheme  (1979,  p.  19). 

Fleck  is  describing  the  difficulty  in  developing  an  historical  document  that  follows  a  chronological  order  yet 
maintains  a  flow  which  keeps  the  reader's  attention.  He  admits  that  many  things  are  happening 
simultaneously.  Ultimately,  it  is  up  to  the  disgression  of  the  historian  and  writer  to  determine  what 
content  is  included  in  the  study  and  what  the  sequence  of  presentation  will  be. 

1  faced  the  same  kind  of  problem  when  organizing  my  history  course.  In  order  to  tell  the  story  of  the 
history  of  educational  technology  and  analyze  that  account,  much  of  the  history  of  the  field  had  to  be  left 
out.  The  course  is  limited  by  necessity.  I  now  analyze  and  interpret  educational  technology  by  looking  at 
the  development  of  the  field  through  two  distinct  yet  related  filters  or  screens.  Looking  at  the  field  of 
educational  technology  through  these  two  screens  limits  the  scope  of  this  course  both  by  necessity  and  by 
definition.  One  simply  cannot  consider  and  give  equal  treatment  to  all  of  the  factors  which  contribute  to  the 
development  of  an  academic  area  of  study  (Fleck,  1979)  in  one  graduate  level  course. 

The  first  of  these  filters  or  screens  was  the  intent  and  effort  to  professionalize  the  audiovisual  field.  Many 
members  of  the  audiovisual  field  were  interested  in  professionalizing  and  gaining  status  for  the  field  but 
James  D.  Finn  spearheaded  this  movement.  Finn  wrote  and  spoke  extensively  about  the  need  for 
audiovisual  specialists  to  become  professionals.  Finn  formed  the  first  academic  department  called 
Instructional  Technology  at  the  University  of  Southern  California.  He  was  instrumental  in  obtaining  large 
amounts  of  funding  to  conduct  conceptual  and  theoretical  studies  of  the  field,  including  the  Technological 
Development  Project  which  funded  the  effort  to  write  the  1963  definition  of  educational  technology.  Finn 
also  sought  the  funding  to  create  an  organization  analogous  to  the  "French  Academy"  for  the  preservation  of 
the  language  inside  the  professional  organization  of  the  field.  The  efforts  to  increase  the  prestige  of  the 
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audiovisual  field  were  political.  The  concepts  and  ideas  which  were  central  to  the  field  of  educational 
technology  were  affected  by  these  politics,  so  politics  and  political  overtones  are  included  in  this  course. 

The  second  of  these  screens  or  filters,  the  influence  of  science  and  engineering  on  the  audiovisual  education 
movement,  is  somewhat  less  political.  The  conceptions  of  science  and  engineering  influenced  the 
interpretation  of  many  of  the  concepts  and  ideas  that  were  central  to  the  field  of  educational  technology  and 
were  included  in  the  field  of  educational  technology.  It  is  true  that  in  some  instances  the  desire  to  gain 
professional  status  gave  reason  to  some  involved  in  the  field  to  adopt  particular  conceptions  of  science  and 
engineering  in  education.  But  there  were  many  instances  when  the  idea  of  gaining  status  for  the  field  was 
not  a  concern  to  individuals  in  the  field.  They  simply  adopted  a  particular  conception  of  science  or 
engineering  based  on  their  individual  belief  system  or  their  academic  background.  As  a  matter  of 
influencing  the  activities  of  the  field  there  was  still  politics  involved  in  these  latter  cases,  but  there  was 
much  less  conscious  thought  and  effort  on  the  part  of  the  members  of  the  field  involved  in  this  level  of 
politics,  decision  making,  and  action. 

Frequently,  particular  ideas  or  conceptions  are  introduced  into  professional  dialogue  for  personal  reasons. 
Overtime  this  motivation  becomes  hidden  as  particular  ideas  grow  in  their  acceptance.  Historical 
investigations  bring  this  hidden  motivation  to  light.  Historians  of  educational  technology  can  contribute  to 
the  self-awareness  of  the  field.  They  can  do  this  by  helping  to  make  the  lost  and  hidden  purposes  into 
conscious  ones.  This  result,  in  turn,  will  open  them  to  a  critical  appraisal  that  may  rekindle  the  discussion 
of  the  moral  and  ethical  responsibility  of  the  professional. 

This  course  is  not  expected  to  provide  any  new  prescriptions  for  action,  nor  is  it  designed  to  predict  what 
any  new  actions  in  the  field  of  educational  technology  might  be.  It  is  undertaken  with  the  intent  of 
analyzing  the  intellectual  basis  for  the  field  of  educational  technology  and  bringing  some  of  the  less 
obvious  ideas  that  are  involved  in  educational  technology  to  a  more  conscious  level. 


Critical  Issues  in  the  Use  of  Technology  in  Education 
GRED  615 

A  good  part  of  the  work  called  "theorizing"  is  taken  up  with  the  clarification  of  concepts- 
and  rightly  so.  It  is  in  this  matter  of  clearly  defined  concepts  that  social  science  research 
is  not  infrequently  defective  (Merton,  1957,  p.  1 14). 

An  often  overlooked  consideration  in  the  theory,  research,  and  practice  in  the  field  of  educational  technology 
is  a  lack  of  conceptual  clarity  and  understanding. 

The  traditional  answer  to  the  problem  of  conceptual  clarification  is  the  commonly  heard  prescription:  define 
terms  precisely.  Educational  technologists  often  do  this  when  they  provide  operational  definitions  in  their 
empirical  studies.  But  it  seems  evident  that  previous  attempts  at  defining  are  at  least  partially  responsible 
for  the  differing  viewpoints  or  conceptions  which  exist  in  the  field  today.  They  are  at  least  partly 
responsible  for  the  difficulty  that  is  encountered  when  trying  to  understand  the  concepts  and  terminology  of 
the  field  that  are  often  open  to  differing  interpretations. 

But  there  is  a  more  fundamental  conceptual  problem.  The  problem  is  the  meaning  of  educational 
technology  itself.  There  have  been  a  number  of  popular  interpretations  of  the  term  "educational 
technology"  since  its  inception.  This  has  caused  some  problems  for  the  field.  The  formal  meaning  of  the 
term  "educational  technology"  was  contested  from  its  earliest  use. 

Educational  technology  has  prided  itself  as  being  an  "applied"  field.  As  such,  its  research  emphasizes  the 
practical  or  prescriptive  aspects  of  instruction.  Historical  studies  focus  on  understanding  their  subjects. 
They  are  not  always  thought  of  as  practical.  Recent  content  analysis  studies  of  major  professional  journals 
and  doctoral  dissertations  in  the  field  of  educational  technology  revealed  that  very  few  conceptual  or 
historical  studies  have  been  conducted  or  published  (Januszewski  &  Young,  1990;  Caffarella,  1992). 

Meaning  is  strongly  tied  to  interpretation.  There  is  a  hesitancy  on  the  part  of  members  of  given  fields  of 
study  to  question  their  own  presuppositions  or  interpretations  about  their  language  habits  (Fleck,  1979). 
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Judging  from  the  number  of  articles  published  in  the  professional  journals  of  the  field  of  educational 
technology,  studies  which  do  focus  on  concepts,  meaning,  and  language  are  not  warmly  received  by  the 
profession. 

Concepts  give  our  experience  direction  and  meaning.  "Our  concepts  structure  what  we  perceive,  how  we  get 
around  in  the  world,  and  how  we  relate  to  other  people"  (Lakoff  and  Johnson,  1980,  p.  3).  This  suggests 
that  the  concepts  that  we  use,  and  the  conceptual  systems  that  we  think  with,  are  the  basis  of  how  we  get 
through  our  daily  lives. 

Lakoff  and  Johnson  described  this  idea  by  saying: 

But  our  conceptual  system  is  not  something  that  we  are  normally  aware  of.  In  most  of 
the  little  things  we  do  every  day,  we  simply  think  and  act  more  or  less  automatically 
along  certain  lines.  Just  what  these  lines  are  is  by  no  means  obvious.  One  way  to  find 
out  is  by  looking  at  language.  Since  communication  is  based  on  the  same  conceptual 
system  that  we  use  in  thinking  and  acting,  language  is  an  important  source  of  evidence 
for  what  that  system  is  like  (1980,  p.  3). 

The  study  of  concepts  can  help  us  to  determine  the  significance  of  events  and  processes  as  well  as  just 
describing  them.  Studying  concepts  can  also  reveal  more  subtle  aspects  such  as  the  emphasis  which  may  be 
placed  on  certain  words  and  actions.  The  subtle  aspects  included  in  certain  concepts  can  affect  professional 
practice. 

Inherent  in  this  last  idea  is  the  connection  between  history  and  concepts.  One  way  to  study  language  is  to 
look  at  its  historical  development.  This  can  be  a  long  and  arduous  process.  But  if  sufficient  evidence  is 
gathered  and  seriously  considered,  great  insight  and  understanding  can  be  attained. 

Some  teaching  strategies  and  considerations 

Reaction  Papers 

For  the  most  part  I  believe  that  it  is  important  that  the  class  sessions  be  guided  by  student  discussion.  I 
still  have  things  that  I  want  to  get  across  to  the  students  but  I  can  build  that  into  a  conversation  rather  than 
make  a  specific  presentation.  In  order  to  prepare  my  students  for  the  first  eight  weeks  or  so  of  these  two 
classes  I  ask  them  to  write  "reaction  papers".  This  is  an  idea  I  adapted  from  Don  Ely  at  Syracuse 
University.  A  reaction  paper  is  a  500  word  maximum,  discussion  of  the  students  thoughts,  opinions,  or 
reactions  to  the  readings  assigned  for  that  day.  It  is  intended  to  be  personal.  Students  sometimes  state  how 
this  reading  made  them  think  about  something  they  never  thought  of  before.  Sometimes  they  can  give  an 
example  from  their  own  lives.  They  are  not  graded,  but  I  "respond"  to  them  by  writing  comments,  notes, 
and  questions  on  them  and  returning  them  the  following  class  session.  These  reaction  papers  are  the 
starting  point  for  the  discussion  in  class  for  that  day.  Reaction  papers  have  several  advantages:  1)  students 
must  come  to  class  somewhat  prepared;  2)  students  usually  'process'  the  information  in  the  readings  and  put 
their  own  personal  spin  on  some  ideas;  3)  all  students  have  an  opportunity  to  organize  and  express  their 
opinion.  Reaction  papers  could  be  used  in  other  settings  and  with  other  purposes  but  I  find  that  this  works 
for  me. 

Seminar  Session  Leadership 

Another  activity  that  has  worked  well  for  me  is  the  student  run  and  operated  seminar  session.  This  tends  to 
work  better  in  the  class  size  is  smaller.  I'm  inclined  to  use  it  more  in  the  critical  issues  class  than  I  am  in 
the  history  class.  There  are  two  ways  that  this  type  of  seminar  session  could  go  and  much  depends  on  your 
(faculty  member's)  level  of  comfort  with  giving  up  control.  First,  there  is  the  type  where  the  faculty 
member  chooses  the  topic  and  the  readings  and  the  students  prepare  for  and  conduct  the  class  session.  Here 
they  run  the  class,  perhaps  much  like  you  would. 

< 

The  second  type  is  where  the  students  select  the  topic  and  the  readings  and  run  the  class  session.  This  can 
be  a  little  more  wide  open  so  be  cautious.  I  like  to  have  the  students  clear  the  topic  and  the  readings  with 
me  in  advance  so  I  can  help  them  select  good  readings  and  be  certain  that  the  topic  fits  into  the  intention  of 


the  course.  More  often  then  not  students  like  to  assign  other  students  lots  of  readings  but  tend  not  to  like 
to  be  assigned  lots  of  readings  by  you.  The  selection  of  readings  based  on  an  intended  goal  is  helpful  to 
student  development.  Group  seminar  leadership  can  work  well. 

Final  Papers 

Most  faculty  members  are  familiar  with  this  idea.  I  tend  to  use  one  of  two  kinds  of  papers.  First,  there  is 
the  paper  that  may  be  the  result  of  a  seminar  session.  Here  the  student  incorporates  and  organizes  much  of 
what  happened  at  the  session.  It  also  includes  additional  research  gathered  after  the  session  to  sharpen 
arguments  and  add  and  support  points. 

The  second  paper  is  the  research  paper.  I  tend  to  use  this  more  in  the  history  class.  Papers  take  one  of 
three  basic  forms:  1)  a  biographical  paper,  2)  a  history  of  some  specific  idea  or  concept  in  the  field,  and  3)  a 
time  limited  history  of  the  field.  These  can  be  difficult  at  the  Masters  level  because  students  have  not  yet 
become  familiar  with  many  of  the  nuances  of  the  Field.  Students  are  encouraged  to  interpret  as  well  as 
describe  activities  in  the  field.  Be  prepared  to  meet  with  students  outside  of  class  time  in  order  to  support 
their  efforts. 
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Purpose 


The  purpose  of  this  presentation  is  to  provide  some  conceptual  frameworks  for  understanding  critics  of 
Educational  Technology.  These  conceptual  frameworks  will  be  de.veloped  and  fleshed  out  by  using  a  series 
of  concepts,  approaches  and  techniques  of  historical  analysis  from  the  area  of  Intellectual  History, 
specifically  the  History  of  Ideas.  We  hope  to  demonstrate  how  studying  history  can  help  members  of  the 
field  of  Educational  Technology  to  gain  a  clearer  understanding  of  the  problems  and  challenges  that  they 
face. 

This  study  should  help  to  answer  the  questions  'why  do  critics  of  educational  technologists  think  the  way 
they  do  about  educational  technology?'  and  'how  has  that  way  of  thinking  changed  with  time?'.  As 
historian  George  Boas  stated,  "the  history  of  ideas  tells  us  among  other  things  how  we  got  to  think  the  way 
we  do-and  if  that  is  not  of  importance  one  wonders  what  is"  (Boas,  1969,  p.  3). 

In  this  study,  understanding  the  critics  of  the  concept  of  educational  technology  is  the  goal.  But 
understanding  has  not  always  been  seen  as  a  useful  end  by  many  of  those  involved  in  the  field  of  educational 
technology  (Yeaman,  1990).  Many  in  the  field  thought  that  an  "understanding"  was  useful  in  so  far  as  it 
was  a  necessary  step  in  accomplishing  some  goal  or  task.  They  considered  it  to  be  a  necessary  step  toward 
explaining  or  predicting  a  desired  outcome  (Koetting,  1983).  Usually  the  larger  goal  was  to  provide  a 
prescription  to  be  used  in  professional  practice.  This  study  is  not  expected  to  provide  any  new  prescriptions 
for  action,  nor  is  it  undertaken  with  the  aim  of  predicting  what  any  new  definition  of  educational  technology 
might  be.  It  is  undertaken  with  the  intent  of  bringing  some  of  the  less  obvious  ideas  that  are  involved  in 
educational  technology  to  a  more  conscious  level. 

In  order  to  fairly  assess  the  critics  of  educational  technology  and  their  history  one  must  decide  on  an 
inception  point  for  the  field  of  educational  technology  in  order  to  begin  the  analysis.  One  could  trace  the 
beginnings  of  educational  technology  back  to  the  educational  practices  of  the  ancient  Greeks,  or  Comenius, 
orPestolozzi,  or  Herbart  (Saettler,  1990).  But  this  could  be  somewhat  misleading  because  much  of  modern 
day  education  has  its  beginnings  in  the  ideas  of  these  individuals.  It  would  be  more  accurate  to  say  that 
educational  technology  is  a  twentieth  century  phenomenon  with  its  roots  in  the  educational  ideas  of  the 
progressive  education  movement  (Shrock,  1990)  and  with  the  onset  of  the  industrial  revolution.  The 
position  taken  in  this  presentation  is  that  it  was  not  so-called  empirical  considerations  applied  to  schooling 
that  led  to  the  construction  and  development  of  the  idea  of  educational  technology.  It  was  the  interpretation 
science  and  industrial  development,  which  were  of  great  social  and  psychological  significance,  that 
contributed  to  the  development  of  the  concept  of  educational  technology  (Kliebard,  1987;  Callahan,  1962). 

Methodological  Concerns 

The  history  of  ideas:  a  description  of  an  umbrella  like  approach 

Studies  in  history  (in  this  case  of  educational  technology)  can  be  viewed  from  differing  domains  of 
intellectual  history  (Higham,  1977).  Intellectual  history  is  the  study  of  the  role  of  human  thought  in 
shaping  the  history  of  some  given  entity  or  occurrence.  Intellectual  history  can  itself  be  analyzed  in  terms 
of  two  contrasting  tendencies  or  methods  that  are  followed  to  investigate  the  role  of  thought  in  history. 
The  essential  difference  between  these  two  methods  has  to  do  with  the  level  of  conscious  thought  that  a 
historian  chooses  to  highlight  (Veysey,  1977).  Studies  in  the  history  of  ideas  often  cross  traditional 
disciplinary  boundaries  seeking  important  or  "great  works",  and  coherent  expressions  of  thought  written 
about  a  topic  in  question.  Studies  in  what  is  called  the  history  of  culture  focus  more  on  the  development  of 
the  group  consciousness  and  the  multiple  topics  or  influences  that  help  to  shape  the  thought  processes  of  a 
group. 

The  difference  between  the  history  of  ideas  and  the  history  of  culture  is  a  matter  of  degree.  There  is  much  to 
be  learned  from  the  study  of  how  consciously  articulated  ideas  become  a  part  of  a  "thought  collective"  in  a 
given  field  of  study  (Fleck,  1979;  Kuhn,  1970).  Although  these  two  approaches  to  intellectual  history  have 
different  purposes,  they  complement  and  support  each  other  (Higham,  1977). 

As  originally  conceived,  the  history  of  ideas  approach  was  both  a  method  and  a  theory  of  history  (Lovejoy, 
1940).  Studies  of  the  history  of  ideas  often  analyzed  some  particular  idea  at  a  singular  point  in  time, 
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exemplified  by  Hollinger's  (1985)  In  the  American  Province:  The  Problem  of  Pragmatism  in  American 
History  .  They  also  could  be  descriptions  of  the  changes  in  the  meanings  of  particular  ideas  over  time,  such 
as  Lovejoy's  (1940)  Great  Chain  of  Being.  As  a  method,  the  history  of  ideas  focused  on  the  "root  ideas" 
(Lovejoy,  1940)  which  constituted  a  broader  concept  or  idea.  This  approach  was  similar  to  the  approach 
used  in  analytic  philosophy  which  developed  in  the  early  twentieth  century.  These  root  ideas  were  often  the 
"necessary  and  sufficient  conditions"  of  the  idea.  Establishing  these  conditions  established  the  boundaries  of 
a  particular  idea.  The  reasoning  was  that  if  the  essential  components  of  an  idea  could  be  understood,  then 
the  meaning  of  the  idea  would  become  clear. 

The  history  of  ideas  is  still  concerned  with  much  of  the  same  subject  matter  as  it  always  has  been,  but  now 
it  seems  to  be  a  much  "looser"  approach  in  which  the  distinction  between  the  history  of  ideas  and  the 
history  of  culture  has  diminished,  as  in  Foucault's  studies  such  as  The  Arcfiaeology  of  Knowledge  (1972) 
and  The  Order  of  Things  (1970).  Studies  under  both  labels  have  been  used  to  examine  the  same  topics.  In 
fact,  many  scholars  believe  that  the  titles  "history  of  ideas"  and  "history  of  culture"  should  be  discarded  in 
favor  of  intellectual  history  (Kelley,  1990).  What  may  be  of  some  interest  to  the  field  of  educational 
technology  is  that  the  history  of  ideas  has  included  both  the  ultra  rational  and  objective  studies  of  Lovejoy 
(where  if  the  proper  analysis  were  conducted  the  truth  would  reveal  itself)  and  the  postmodern  work  of 
Foucault  (where  he  tried  to  discredit  any  a  priori  method). 


Potential  problems  with  the  approach 


There  are  at  least  four  potential  difficulties  with  the  way  that  the  history  of  ideas  is  conducted  as  a  research 
method.  First,  the  necessary  and  sufficient  conditions  approach  has  been  called  "atomistic"  (Mink,  1968). 
The  criticism  here  is  that  by  concentrating  on  the  component  parts  or  root  ideas  of  a  particular  concept,  the 
"whole"  of  the  concept  or  idea  is  neglected.  It  is  argued  that  while  it  is  important  to  identify  the 
components  of  an  idea,  it  is  the  interaction  of  these  components  or  root  ideas  that  give  meaning  to  the 
whole  of  the  idea  under  study.  The  atomistic  nature  of  the  traditional  method  of  the  history  of  ideas 
relegates  this  interaction  to  a  secondary  status.  This  criticism  may  be  characterized  by  the  conventional 
logic  that  "the  whole  is  greater  than  the  sum  of  its  parts",  a  notion  that  is  associated  with  systems  theory  in 
the  field  of  educational  technology. 

Second,  it  has  been  suggested  that  the  traditional  approach  results  in  an  elitist  perspective  (de  Carvalho, 
1988).  The  written  record  of  the  past  is  limited.  By  and  large,  the  records  that  exist  today  were  written  by 
people,  most  often  males,  who  were  either  wealthy  or  were  afforded  the  special  opportunity  to  record  their 
thoughts  and  observations.  In  proportion,  very  few  documents  exist  that  were  produced  by  "commoners". 
Commoners  simply  did  not  have  the  opportunity  to  contribute  to  the  literature  in  the  way  that  elites  could. 
This  resulted  in  a  written  record  which  was  largely  produced  by  elites.  Focusing  solely  on  this  written 
record  could  result  in  a  limited  perspective  on  the  subject  being  studied  if  this  limitation  is  not  properly 
acknowledged  in  advance. 

Third,  it  has  been  noted  that  the  traditional  approach  is  not  sufficiently  concerned  with  matters  of  context. 
While  it  is  true  that  the  purpose  of  this  method  is  to  understand  the  context  in  which  a  term  is  used  or  an 
idea  is  expressed,  it  has  not  often  acknowledged  that  the  investigator  is  also  operating  within  a  specific 
context:  the  present.  Without  properly  considering  the  context  in  which  the  investigator  is  working,  the 
outcome  of  a  particular  study  may  not  be  "objective"  (Mink,  1968).  Due  in  part  to  these  criticisms  the 
thought  that  the  history  of  ideas  is  a  singular  methodology  seems  to  have  faded.  The  history  of  ideas  is 
now  more  frequently  thought  of  as  a  form  of  intellectual  history  (Higham,  1977;  Vesey,  1977). 

Finally,  the  anti  methodological  stance  taken  by  Foucault  has  brought  the  entire  notion  of  objectivity  into 
question.  Without  any  possibility  of  an  objective  history  the  utility  of  historical  projects  has  been  called 
into  serious  question. 

There  is  no  single  correct  way  to  conduct  a  study  in  the  history  of  ideas  (the  history  of  ideas  is  to 
historical/conceptual  research  what  survey  research  is  to  quantitative  research  -  there  are  many  factors  which 
are  specific  to  individual  accounts).  The  history  of  ideas  has  become  an  umbrella  concept  where  the  specific 
procedures  of  investigation  and  analysis  arc  defined  differently  in  each  study.  Many  historians  who  work  in 
this  area  agree  that  there  are  several  important  factors  to  consider  when  conducting  a  study  in  the  history  of 
ideas. 
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Five  factors  in  the  history  of  ideas 


1 )  Have  precise  boundaries  for  the  idea  being  studied  (Lovejoy,  1940;  Boas,  1969;  Kelley,  1990). 

2)  Be  aware  of  the  context  in  which  the  idea  is  being  studied.  Meanings  of  words  and  the  ideas  that 
they  connote  change  with  time  (Lovejoy,  1940;  Boas,  1969;  Fleck,  1979;  Kelley,  1990). 

3)  Be  prepared  to  cross  traditional  disciplinary  boundaries  when  studying  an  idea  (Lovejoy,  1940; 
Boas,  1969;  Kelley,  1990). 

4)  Be  aware  of  the  'metaphysical  pathos',  the  attitude  a  writer  portrays  toward  the  idea.  The  language 
used  by  the  writer  can  show  an  idea  in  a  positive  or  negative  light  (Lovejoy,  1940;  Boas,  1969). 

5)  Remember  that  the  history  of  ideas  is  not  confined  to  semantics.  Ideas  often  have  more  than  one 
name  (Lovejoy,  1940;  Boas,  1969). 

Many  analyses  now  fit  under  the  umbrella  of  the  history  of  ideas. 
Definitions  and  historical  studies 

Definitions  are  important  when  studying  history.  They  are  especially  important  when  conducting  studies  in 
the  history  of  ideas  because  definitions  often  set  the  boundaries  for  a  study.  Historian  David  Fischer  (1971) 
identified  fifteen  kinds  of  definitions  that  historians  have  frequently  used  in  their  work.  He  reasoned  that 
using  different  kinds  of  definitions  in  studies  may  yield  different  results. 

There  are  many  different  kinds  of  formal  definition.  A  historian  ought  to  choose 
consciously  and  carefully  from  the  range  of  possibilities  available  to  him.  Imprecision 
results  not  merely  from  an  incomplete  or  inaccurate  or  inconsistent  definition,  but  also 
from  the  use  of  an  inappropriate  definitional  type.  It  could  be  one  or  more  of  the 
following  types,  some  of  which  overlap  (p.  277). 

Of  the  fifteen  definitional  types  posited  by  Fischer,  there  are  four  that  are  of  particular  interest  for 
this  presentation  on  educational  technology.  They  are: 

1 .  A  definition  by  genus  and  difference  locates  a  term  within  a  larger  class,  and  then 
supplies  specific  differences  (p.  277). 

2.  A  theoretical  definition  might  include  a  statement  of  principles  involved  in  an 
idea  (p.  278). 

3.  An  analytical  definition  defines  a  thing  by  detailing  its  parts  (p.  279). 

4.  A  stipulative  definition  introduces  a  wholly  new  expression  into  the  language  or 
gives  a  new  and  special  meaning  to  an  old  expression  (p.  278). 

The  Presentation  Format 

Acknowledgment  of  limitations 

There  are,  of  course,  a  number  of  ways  in  which  the  criticisms  of  Educational  Technology  could  be 
conceptualized  or  grouped.  Certainly  too  many  to  be  discussed  in  one  presentation.  It  is  also  likely  that 
many  of  the  possible  conceptual  frameworks  for  analyzing  these  criticisms  would  overlap.  That  is,  certain 
individual  criticisms  could  fit  in  more  than  one  framework  for  analysis. 
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The  Format 


This  presentation  will  revolve  around  the  following  conceptual  frameworks  for  analyzing  the  criticisms  of 
Educational  Technology:   1)  concerns  about  the  development  of  man-machine  systems;  2)  concerns  about 
technological  momentum;  3)  concerns  about  individualizing  instruction;  4)  concerns  about  gender 
inequities  and  bias;  and,  5)  concerns  about  dehumanization. 

A  summary  of  the  highlights  and  findings  of  each  of  the  five  papers  follows. 

The  analysis  of  man-machine  systems  centered  on  the  relationship  between  people  and  machines, 
specifically  their  interactions  and  human  self-image.  The  major  points  identified  in  this  paper  include: 

1)  Scientific  inquiry  and  technological  development  leave  a  void  in  the  spiritual  and  artistic  side  of  people 
which  needs  to  be  expressed. 

2)  Technology,  as  an  extension  of  people,  has  threatened  to  sever  humaii  wholeness  by  separating  the 
cognitive  (mind)  from  the  psychomotor  (body). 

3)  Engineers  (and  those  who  view  education  as  engineering)  form  and  develop  their  moral  standards  based 
on  the  things  that  they  build  rather  than  for  and  from  the  society  for  which  they  build  them. 

The  analysis  of  technological  momentum  encompasses  the  following  ideas:  using  technology  because  it  is 
there;  the  implementation  of  technology  requiring  the  need  for  more  technology;  and  the  deskilling  of 
teachers  because  of  the  ever  increasing  use  of  technological  design.  The  major  points  identified  in  this 
paper  include: 

1 )  That  the  are  2  kinds  of  technological  momentum:  a)  that  there  is  too  much  technology  in  education,  and 
b)  that  technology  is  being  introduced  into  education  too  fast. 

2)  Technological  evolution  in  education  has  accelerated.  In  some  instances  this  has  resulted  in  the 
deskilling  of  teachers. 

3)  One  of  the  major  facets  of  technology  is  the  concept  of  control.  Critics  of  educational  technology  have 
argued  that  it  is  the  technology  which  controls  teachers,  learners  and  the  learning  process  rather  than 
teachers  and  learners  controlling  technology.  The  implementation  of  technological  developments  needs  to 
be  controlled  and  not  left  to  chance. 

The  analysis  of  individualized  instruction  addresses  the  criticisms  of  self-instructional  systems,  including; 
programmed  learning,  computer  assisted  instruction,  and  independent  studies.  The  major  points  identified  in 
this  study  include: 

1)  Individualization  of  instruction  has  meant  segmenting  and  providing  self  paced  approaches  for 
individuals  to  learn  the  same  material.  It  does  not  mean  the  selection  of  different  material  or  subject  matter 
for  each  learner. 

2)  Individualization  maintains  the  status  quo  by  maintaining  the  power  and  authority  of  the  teacher  in  an 
"individualized"  environment  rather  than  a  group  paced  one. 

3)  Individualization  in  education  threatens  the  group  and  social  aspects  of  education. 

The  analysis  offender  inequities  and  bias  will  include  focus  on  computer  utilization  from  1983-1993.  The 
major  points  identified  in  this  paper  include: 

1)  Criticisms  focus  of  issues  involving  content,  form,  and  function.  The  issue  of  content  examines  the 
design  and  subject  matter  of  software.  Form  is  concerned  with  the  machine  hardware.  And  function  is 
related  to  issues  of  access. 
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2)  Critics  use  qualitative,  quantitative,  and  conceptual  methodologies  to  show  gender  bias  in  the  use  of 
computers  in  education.  Recent  research  suggests  examination  of  the  issue  from  a  societal  perspective 
rather  than  a  technological  one. 

3)  Educational  computing  experiences  must  allow  for  differing  ability  and  interest,  as  well  as  different 
types  of  interactions. 

The  analysis  of  dehumanization  will  address  the  following:  alienation  because  of  machine  implementation, 
and  the  deskilling  of  students  and  teachers  because  of  the  introduction  of  technological  innovations.  The 
major  points  identified  in  this  paper  include: 

1 )  Technology  has  been  represented  as  being  value  neutral.  Critics  challenge  this  idea  because  there  are 
values  inherent  within  the  technology,  such  as  efficiency,  machine  utilization  etc.. 

2)  Technological  expansion  has  affected  the  language  that  is  used  in  education.  The  metaphorical  structure 
of  language  leads  to  seeing  the  world  in  technological,  rather  than  otherterms. 

3)  Teachers  are  often  viewed  as  managers  and  facilitators  rather  than  educators.  Examinations  of  values 
associated  with  education  question  the  desirability  of  this  view. 

Concluding  comments  and  summation 

If  the  criticisms  of  the  concepts,  principles  and  implications  of  educational  technology  can  be  traced  back  in 
the  literature  of  the  nineteenth  century  then  it  seems  reasonable  to  think  that  the  critics  of  the  ideas  behind 
educational  technology  have  been  around  as  long  as  these  ideas  which  are  important  to  educational 
technology  have  been. 

I  would  like  to  make  three  points  in  closing.  First,  if  Bob  Heinich  is  right,  and  educational  technology  is  a 
part  of  the  larger  idea  of  technology  (1984),  then  it  seems  reasonable  to  think  that  many  of  the  criticisms  of 
educational  technology  will  be  anchored  in  criticisms  of  the  larger  idea  of  technology.  And,  in  fact,  this  is 
accurate.  Commentary  on  technology  dates  back  to  the  first  (or  the  early  part  of  tine)  industrial  revolution. 
This  commentary,  analysis,  or  criticism  seems  to  take  one  of  two  basic  forms.  It  appears  as  if  one  form  is 
part  of  the  world  of  literature  like  Shelley's  Frankenstein  or  Vonnegut's  Player  Piano.  Jim  Finn  had  run 
ins  with  Griswold  and  other  "humanities  types".  The  other  basic  form  is  a  more  scholarly/academic  or 
social  science  based  approach.  This  group  can  be  represented  by  the  work  of  Karl  Marx,  Lewis  Mumford, 
and  Jacques  Ellul.  There  are  many  more  commentaries  in  the  literature  that  follow  a  social  science  based 
approach.  Both  groupings  of  these  critics  tend  to  focus  on  the  areas  of  "the  human  condition",  the  meaning 
of  progress,  and  moral  questions  about  technology. 

The  second  thing  is  that  there  are  a  good  many  critics  of  educational  technology  represent  other  perspectives 
in  the  broader  field  of  education.  Michael  Apple  and  Henry  Giroux  are  two  of  many  important  writers  in 
this  area.  While  these  critics  raise  many  questions  it  seems  to  me  that  they  all  include  the  same  base 
notion.  That  educational  technologists  and  others  may  have  fundamental  differences  in  the  purposes  and 
meaning  of  the  concept  of  education,  and  in  the  processes  involved  in  bringing  those  purposes  to  reality.  A 
serious  conceptual  question  about  education. 

Third,  as  the  field  matured,  a  number  of  criticisms  were  posed  by  individuals  within  the  field.  These 
criticisms  included  moral,  epistemological,  and  procedural  questions.  These  criticisms  were  the  result  of 
different  interpretations  of  the  concept  of  educational  technology  within  the  field  itself.  This  resulted  in 
alternative  priorities,  plans,  and  paradigms  for  educational  technology  (e.g..  Eraut,  1985). 


There  have  been  many  criticisms  leveled  against  the  ideas  and  use  of  Educational  Technology.  This 
presentation  will  have  served  as  a  way  to  see  the  relationships  between  various  approaches  or  conceptual 
frameworks  of  the  criticisms  of  the  field.  Understanding  the  various  criticisms  and  their  relationships  to 
each  other  is  essential  to  moving  our  field  forward. 
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The  improvement  of  the  educational  system  and  the  improvement  of  the  act  of  teaching  is  a 
common  goal  of  all  educators.  Increases  in  class  size,  time  constraints,  and  class  material  have  pressured 
educators  to  develop  better  ways  to  present  instructional  material.  This  pressure  has  led  to  numerous 
teaching  innovations,  including  many  in  the  area  of  Individualized  Instruction. 

Examples  of  Individualized  Instruction  can  be  seen  as  far  back  as  ancient  Greece  with  the  teachings 
of  Socrates,  and  recently  in  the  development  of  sophisticated  Computer  Managed  Instruction  in  the  1980's 
and  1990's.  Examples  of  programs  of  individualized  instruction  in  elementary,  secondary,  and  continuing 
education  have  increased  throughout  the  twentieth  century,  as  the  demand  for  new  teaching  innovations  has 
increased.  As  in  the  case  with  any  educational  innovation  there  has  been  a  substantial  amount  of  review  of 
these  programs,  both  of  the  quality  of  instruction  delivered  and  of  their  foundations  in  theory.  These 
reviews  have-been  both  positive  and  negative. 

This  paper  concentrates  on  a  history  of  the  critiques  of  Individualized  Instruction  in  the  twentieth 
century,  especially  form  1960  to  present.  Three  examples  of  Individualized  Instruction  will  be  specifically 
covered,  these  include:  Audio-Tutorial  by  S.  N.  Postlethwait,  Personalized  System  of  Instruction  by  Fred 
Keller  (PSI),  a;. J  Computer  Assisted  Instruction  (CAI).  Recent  trends  towards  Computer  Managed 
Instruction  through  Instructional  Development  were  not  included  in  this  paper.  These  three  examples  were 
chosen  based  on  their  importance  to  present  day  education..  Popularity  does  not  in  itself  make  something 
lasting  and  sound.  Widespread  acceptance  of  technique  without  sound  evidence  for  that  technique  is  just  the 
formula  for  producing  another  "educational  fad"  (Powers,  1972,  p.4). 

The  discussion  of  each  type  of  Individualized  Instruction  will  include:  a  brief  description  of  the 
theory  and  methods,  a  summary  of  the  benefits  as  indicated  by  the  designers,  an  account  by  the  critics,  and  a 
response  by  the  supporters  is  applicable.  A  final  section  addressing  general  critiques  of  Individualized 
Instruction  will  be  included  at  the  end  of  the  document. 


Audio  Tutorial 


Audio-Tutorial  is  a  method  of  Individualized  Instruction  developed  by  S.  N.  Postlethwait  in  1961 
at  Purdue  University.  His  purposes  weie  to  find  an  improved  method  of  teaching  botany  to  a  larger  number 
of  college  students  and  to  effectively  assist  the  students  who  possessed  only  limited  backgrounds  in  the 
subject  (Snortland,  1982,  p.3).  The  development  of  an  A-T  program  requires  a  significant  amount  of 
planning  and  time  by  the  instructor  before  the  course  is  implemented.  Although  there  is  some  room  for 
modification  for  specific  programs  to  be  taught,  the  general  principles  remain  the  same.  The  studsnt  has 
access  to  a  taped  presentation  of  a  specifically  designed  program  that  directs  their  activities  one  at  a  time. 
The  criteria  for  an  effective  educational  program  that  Postlethwait  used  in  designing  A-T  are  repetition, 
concentration,  association,  unit  steps,  use  of  the  communication  vehicle  appropriate  to  the  objective,  use  of 
multiplicity  of  approaches,  and  use  of  an  integrated  experience  approach  (Couch,  1983,  p.6). 

There  are  many  benefits  of  Audio-Tutorial  as  described  by  Postlethwait  (1972),  these  include:  an 
emphasis  on  student  learning  rather  than  on  teaching,  self-pacing,  allowing  better  students  to  accelerate,  not 
having  students  distracted  by  each  other,  more  individual  attention  if  desired,  more  students  accommodated 
in  less  laboratory  space  with  less  staff,  increased  responsibility  of  learning  on  the  students,  and  an  easy 
standardization  of  instruction. 

Some  of  the  major  criticisms  that  are  common  to  Audio-Tutorial  courses  were  illustrated  by 
Snortland  (1982,  p4)  upon  evaluating  a  course  in  Graphics  design. 

1)  "Generally,  the  students  with  previous  drawing  experience  were  able  to  endure  and  prosper 
from  the  individualized  approach  and  enjoy  it  to  a  greater  degree  than  those  who  began  the 
course  with  little  or  not  prior  training  in  the  graphics  language." 

2)  "Self  Pacing  was  a  definite  problem. ...many  students  were  not  ready  to  master  the  additional 
self-discipline  required  in  order  to  maintain  a  steady  pace." 

3)  "...the  A-T.. .approach  is  not  for  everyone." 

The  first  of  these  criticisms  deals  with  background  knowledge,  while  the  last  two  deal  with 
responsibility.  Some  students  respond  to  the  responsibility  placed  upon  them,  while  others  do  not. 
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Snortland  (1982,  p. 5)  explains  "Since  many  freshmen  students  are  not  ready  for  additional  self-discipline 
required  of  them  in  the  A-T  format,  the  choice  of  either  a  structured  approach  or  an  individualized  approach 
should  always  remain  open...". 

Another  criticism  of  A-T,  especially  from  modern  instructional  developers,  is  that  A-T  places  to 
heavy  an  emphasis  on  teacher  control.  All  of  the  material  and  the  learning  and  feedback  procedures  are 
dictated  by  the  instructor.  The  criticism  is  that  this  is  a  severe  form  of  teacher  control  over  the  student. 

Still  further  criticism  deals  with  the  fear  of  the  machine  taking  from  man  the  position  of 
instructor.  Snortland  (1982,  p.5)  replies  to  this  "Man  vs.  Machine"  argument: 

There  is  a  danger  with  this  teaching  method  in  thinking  that  the  machines  can  pretty  well 
take  over  and  thereby  reduce  the  need  for  well  qualified  and  professional  teachers.  Even 
though  an  efficient,  workable,  and  effective  teaching  program  is  fairly  well  in  place  and 
de-bugged,  it  will  still  be  essential  to  have  a  core  of  dedicated  teachers  around  to  keep  the 
system  oiled  and  do  the  many  "in  house"  things  necessary  to  maintain  the  credibility  and 
integrity  of  the  system  as  originally  designed.  There  are  so  many  factors  which  affect  the 
motivation  of  the  students,  checkers,  tutors,  and  teachers,  that  the  machines  by 
themselves,  without  human  support,  would  not  only  be  ineffective,  but  very  likely 
disastrous. 

Another  criticism  is  that  there  is  a  high  initial  dropout  rate  of  students  enrolled  in  A-T  courses. 
The  problem  centered  around  the  fact  that  some  people  were  not  prepared  to  take  on  the  amount  of 
responsibility  that  was  required  of  them  in  order  to  complete  the  course.  This  is  an  even  bigger  problem 
when  one  considers  that  the  courses  most  likely  to  need  an  A-T  setup  are  predominately  courses  with  high 
enrollments  and  a  specific  amount  of  material  to  be  covered  for  the  semester,  a  majority  of  these  courses 
are  freshman  courses  to  begin  with.  This  makes  this  problem  self  defeating  if  not  addressed. 

As  is  the  case  with  most  forms  of  individualized  instruction,  Audio-Tutorial  can  be  modified  to 
meet  the  needs  of  a  particular  situation.  If  effectiveness  were  the  only  criteria  measured,  then  the  conclusion 
regarding  A-T's  effectiveness  compared  to  conventional  instruction  favors  S-T  by  two  to  one  (Couch,  1979, 
p.3).  One  must  be  careful  to  measure  both  the  benefits  and  the  consequences  of  an  A-T  program  before 
implementation. 

Personalized  System  of  Instruction 

The  Personalized  System  of  Instruction,  also  know  as  the  Keller  Plan,  was  introduced  by  Dr.  Fred 
Keller  in  1964.  The  Keller  Plan  is  based  on  10  accepted  educational  principles:  active  responding,  positive 
conditions  and  consequences,  specification  of  objectives,  organization  of  material,  mastery  before 
admancement,  evaluation/objective  congruence,  frequent  evaluation,  immediate  feedback,  self-pacing,  and 
personalization  (McGraw,  1975,  p.4). 

The  five  basic  features  of  the  Keller  plan  are:  self-pacing,  unit  mastery,  student  tutors,  optional 
motivational  lectures,  and  learning  from  written  material  (Couch,  1983,  p.7).  The  design  of  a  course  using 
Keller's  Personalized  System  of  Instruction  consists  of: 

...breaking  the  material  of  the  course  into  several  units  It  entails  dividing  the  material 

into  units  one  to  two  weeks  long  As  each  unit  of  material  is  covered,  specific 

learning  objectives  are  given  to  the  students.  These  state  exactly  what  a  student  must 
know  to  pass  a  unit  quiz  (Grasha,  1977,  p. 8). 

The  proper  implementation  of  a  PSI  Keller  plan  can  be  neatly  divided  into  five  identifiable  states: 
assessing  entering  behaviors,  specifying  objectives,  selecting  resources  and  activities,  establishing  and 
implementing  the  course  framework,  and  evaluating  student  performance  (McGraw,  1975,  p.6). 
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Some  of  the  benefits  of  a  Personalized  System  of  Instruction  include: 

1)  "PSI  students  perform  better  on  all  types  of  examination.  PSI  students  also  demonstrate 
longer  retention  of  material  than  students  in  more  conventional  courses"  (McGraw,  1975, 
p.  16). 

2)  Students  report  "...learning  more  than  do  students  in  a  conventional  course"  (Grasha,  1977,  p. 
7). 

3)  "...student  responses  in  the  affective  domain  are  positive  toward  the  PSI  approach.",  "students 
have  more  positive  attitudes  towards  the  course"  (McGraw,  1975,  p.  16). 

4)  PSI  can  "...increase  the  motivatior  for  further  learning"  (Grasha,  1977,  p.7). 

5)  PSI  students  receive  "...a  much  larger  proportion  of  higher  grades,  despite  controls  for  grading 
criteria."  (McGraw,  1975,  p.20). 

6)  "Keller  Plan  students.. .study  more  per  week  than  do  other  students.. .and  they  do  better  on 
achievement  tests  than  do  traditionally-taught  student"  (Grasha,  1977,  p.8). 

Given  the  large  amount  of  benefits  of  PSI,  there  have  been  an  equally  large  number  of  criticisms. 
Many  of  the  criticisms  are  common  among  many  types  of  individualized  instruction.  The  following  are 
criticisms  aimed  specifically  at  the  Keller  Plan,  yet  many  could  apply  to  Individualized  Instruction  as  a 
whole. 

t 

A  first  criticism  is  that  there  are  many  types  of  proven  instructional  strategies  that  deal  with  large 
groups.  Because  of  the  very  nature  of  PSI,  these  are  not  possible  (Couch,  1983,  p.4). 

A  second  criticism  is  that  there  is  a  higher  dropout  rate  in  the  PSI  programs.  If  a  student  falls 
behind  early  in  the  program,  the  likelihood  that  he/she  will  drop  out  because  they  feel  that  there  is  no 
chance  to  catch-up  with  the  rest  of  the  class.  As  stated  by  Born  &  Moore  (1978,  p.2): 

"That  PSI  causes  or  contributes  to  student  procrastination  has  been  effectively  argued". 
Their  criticism  is  that  procrastination  goes  unnoticed  because  students  are  not  monitored  often  enough. 
Conyers,  Spencer,  and  Sanches  Sosa  (1975,  p.5)  reported  that  incentives  for  completion  of  work  on  time  or 
in  advance  increased  the  students  performance  in  the  course. 

A  third  criticism  is  the  concern  for  students  entering  a  new  course  is  a  negative  attitude.  Because 
of  initial  student  apprehension  towards  a  personalized  instructional  program,  many  students  do  not  develop 
the  sufficient  desire  to  continue  learning  the  material  upon  completion  of  the  program  (Couch,  1983,  p.6). 

A  fourth  criticism  is  that  there  is  a  lot  of  preparation  time  that  goes  into  any  Personalized  System 
of  Instruction  program.  As  Couch  (1983,  p.7)  stated,  "educators...will  find  that  the  developmen  t  of 
materials,  tests,  etc.,  takes  an  inordinate  amount  of  their  time. 

A  fifth  criticism  is  that  the  Keller  Plan  is  based  upon  Skinnerian  conditioning.  The  criticisms  of 
Behaviorism  itself  can  be  in  turn  applied  to  PSI  (Couch,  1983,  p.8). 

A  sixth  criticism  is  that  the  Keller  Plan  decreases  human  interaction.  As  Couch  ( 1983,  p.89) 
states,  "...instructors  might  feel  alienated  from  their  own  courses  after  setting  them  up  to  run  without  the 
need  of  lectures." 

A  seventh  criticism  came  from  Keller  himself.  He  stated  that  the  administration  may  object  to  a 
new  system  that  allows  the  instructor  to  "escape"  their  lecturing  responsibilities  (Couch,  1983,  p.9). 

A  final  criticism  is  that  "Grade-flation"  will  most  certainly  come  up  as  a  controversial  point 
(Sherman,  1976,  p.4).  This  refers  to  the  fact  t  students  who  complete  a  Keller  course  generally  receive  an 
A  for  completing  the  course,  or  do  not  complete  the  course  or  get  a  grade. 

There  are  many  responses  to  these  criticisms.  In  response  to  the  criticism  that  there  is  not  a 
positive  attitude  entering  the  program,  pi  oponents  state  that  if  the  Instructor  makes  an  effort  to  be  positive 
and  enthusiastic  towards  the  course,  this  enthusiasm  will  carry  over  to  the  students.  By  the  time  they  leave 
the  course,  this  initial  negative  attitude  that  a  student  might  have  had  will  be  long  forgotten. 
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The  criticism  that  there  is  too  much  preparation  time  is  answered  by  the  claim  that  there  is  equal 
time  gained  in  the  course.  Grasha  (1977,  p.4)  wrote:  "In  trade  for  the  initial  time-investment,  time  will  be 
freed  during  the  course  from  lectures  and  demonstrations." 

People  in  support  of  the  Keller  Plan,  interestingly  enough,  responded  to  the  accusations  that  PSI 
was  based  on  Skinnerian  By  claiming  that  Behaviorism  is  part  of  education  and  should  be  a  part  of  PSI. 
For  instance:  "This  attitude  (negativity  towards  PSI  based  on  Skinnerian  principles),  however,  overlooks 
the  conditioning  nature  of  education,  regardless  of  the  type  of  instructional  technique  utilized"  (McGraw, 
1975,  p.  12). 

The  debate  over  the  effectiveness  of  Keller's  Personalized  System  of  Instruction,  with  its 
advantages  and  disadvantages  has  been  a  predominant  theme  in  the  literature  on  Individualized  Instruction  for 
the  last  25  years.  There  are  indeed  instances  where  a  Personalized  System  of  Instruction  would  be  more 
beneficial  than  a  conventional  lecture  class.  This  would  apply  especially  to  classes  in  which  enrollment 
was  high,  course  material  was  standardized,  and  faculty  resources  were  scarce.  On  the  other  hand,  when 
there  is  not  a  shortage  of  faculty,  and  the  class  is  not  a  high  enrollment  class,  the  course  would  better  be 
taught  with  more  conventional  methods,  yet  still  based  on  sound  educational  principles. 

Computer  Assisted  Instruction 

The  potential  for  computer  assisted  instruction  was  realized  long  before  the  home  computer  was 
technologically  possible.  It  is  difficult  to  say  when  CAI  was  first  developed,  but  there  are  some  early 
examples  of  it.  The  potential  for  Individualized  Instruction  through  CAI  was  realized  by  John  E.  Coulson. 
He  pointed  out  this  potential  in  the  article,  "Computer-Assisted  Instruction  and  Its  Potential  for 
Individualizing  Instruction"  (1973,  p. 3).  Coulson  wrote:  "A  modern  computer  has  characteristics  that 
closely  parallel  those  needed  in  any  educational  system  that  wishes  to  provide  highly  individualized 
instruction." 

The  benefits  that  Coulson  ( 1970,  p. 3)  saw  from  the  computer  were: 

1)  "...it  has  a  very  large  memory  capacity  that  can  be  used  to  store  instructional  content  material 
or.. ..to  generate  such  material, 

2)  "The  computer  can  perform  complex  analyses  of  student  responses  by  keyboard,  punched 
cards,  electronic  pen,  or  other  techniques  into  the  computer." 

3)  "The  computer  can  make  decisions  based  on  the  assessments  of  student  performance,  matching 
resources  to  individual  student  needs"  (Coulson,  1970,  p.4). 

Other  benefits  in  the  area  of  software  that  might  or  might  not  have  seemed  a  possibility  in  1970 
include  digitization  of  speech  and  video,  work  prediction  software,  alternative  keyboards,  and  switches  with 
appropriate  software.  Computer  Assisted  Instruction  has  come  farther  in  dealing  with  handicapped  and 
disabled  students  than  possibly  could  have  been  realized  at  the  time.  These  benefits  and  more  were  borne 
out  later,  yet  for  every  advantage  of  a  tool,  one  can  usually  find  a  disadvantage. 

Some  of  the  side  effects  of  computer  assisted  instruction  are  stated  by  Henry  F.  Olds  in  an  article 
entitled  "The  Microcomputer  and  the  Hidden  f  urriculum'>  Olds  criticized  Computer  assisted  instruction  for 
its  hidden  side-effects.  Some  of  these  include: 

1)  "Learning  is  in  control  of  some  unknown  source  that  determines  almost  all  aspects  of  the 
interactive  process.  To  learn  one  must  suspend  all  normal  forms  of  interaction  and  engage  only  in  those 
called  for  by  the  program"  (Olds,  1985,  p.5). 

2)  "Learning  is  an  isolated  activity  to  be  carried  on  primarily  in  a  one-to-one  interaction  with  the 
computer.  Normal  inter-human  dialogue  is  to  be  suspended  while  learning  with  the  computer"  (Olds,  1985, 
p.5). 

3)  "Learning  involves  understanding  (psyching  out)  iiow  the  program  expects  one  to  behave  and 
adapting  one's  behavior  accordingly.  One  must  suspend  idiosyncratic  behavior"  (Olds,  1985,  p.5). 

Some  of  these  criticisms  were  answered  later  by  Olds  (1985,  p.6)  when  he  stated:  "...time  on-line 
needs  to  be  mixed  with  plenty  of  opportunities  for  human  interaction."  and  CAI  should  allow  people  to 
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"...jump  around  within  the  program  structure...."  These  partially  answer  the  criticisms  about  decreased 
human  interaction  and  a  lack  of  room  for  creativity. 

Computer  Assisted  Instruction  is  quickly  becoming  the  forerunner  in  Individualized  Instruction. 
As  home  computers  become  more  powerful  and  less  expensive,  the  possibilities  grow  larger  and  larger. 
However,  many  of  the  criticisms  will  not  go  away  with  improved  technology.  There  needs  to  be  an 
improvement  in  the  design  of  the  software  used,  as  well  as  an  improvement  in  the  methodology  used  to 
implement  computers  into  education,  specifically  in  school  reform.  Computer  Managed  Instruction 
through  Instructional  Development  is  the  most  recent  trend  in  school  reform.  It  addresses  the  need  of 
school  systems  and  offers  solutions  through  instructional  development,  an  analysis  of  the  learners,  the 
environment,  economics,  and  instructors  is  conducted  to  best  prescribe  a  program  that  will  match  the  needs 
of  each  individual  situation. 

General  Critiques 

Critiques  are  usually  focused  on  one  particular  variety  of  Individualized  Instruction.  Frequently, 
however,  they  focus  their  attack  at  the  broader  concept  itself.  While  the  characteristics  of  Individualized 
Instruction  that  are  criticized  are  generally  applicable  to  all  types  of  Individualized  instruction,  some  are  not. 
Responses  by  the  different  strands  of  Individualized  Instruction  whom  the  criticism  does  not  apply  are 
abundant.  They  are  quick  to  point  out  to  the  critics  that  not  all  individualized  instruction  is  lie  this. 

The  article  "Individualization:  The  Hidden  Agenda",  by  Ronald  T.  Hyman  is  an  example  of  one 
such  critique.  What  Mr.  Hyman  is  concerned  with  are  the  latent  functions  of  Individualization.  He  is  aware 
of  and  gives  examples  of  two  of  the  manifest  function,  the  segmentation  of  material  and  student  self-pacing. 
He  criticizes  the  segmentation  of  material  by  writing  "Segmented  Junk  is  Still  Junk"  (Hyman,  1973,  p.2). 
His  point  is  that  in  the  push  for  individualization,  many  peoples  definition  is  to  divide  the  subject  matter 
up  into  segments  and  teach  it  at  a  self  taught  level.  There  is  no  concern  for  what  is  really  the  problem,  and 
that  its  the  subject  matter  itself.  He  claims  that  what  individualization  does  not  do  is  to  alter  the  subject 
matter  based  on  the  needs  of  the  student.  Without  doing  this,  there  is  a  compromise  of  Individualized 
Instruction.  The  concept  of  individualization  that  he  offers  should  be  concerned  with  releasing  the  potential 
of  the  student.  To  do  this,  there  must  be  an  "...emphasis. ..on  who  the  pupil  is,  what  he  can  become,  and 
how  he  interacts  with  people  and  objects  around  him"  (Hymari,  1973,  p.4). 

Other  criticisms  that  he  has  with  the  current  usage  of  Individualized  Instruction  are: 

1     "Individualization.. .maintains  the  status  quo.  The  power  i;nd  authority  of  the  teacher  are  key 
aspects  of  the  status  quo"  (Hyman,  1973,  p.5). 

2)  "With  the  isolation  of  the  pupil  comes  the  loss  of  group  camaraderie"  (Hyman.  1973,  p.5) 
stemming  from  #2  above  are: 

3)  a  loss  of  "...the  very  essence  of  democracy... the  feeling  of  responsibility  to  our  fellows" 
(Hyman,  1973,  p.5) 

4)  "...the  loss  of  group  interaction  leads  to  the  minimizing  of  peer  teaching  from  which,  both 
peer  teacher  and  student  would  benefit"  (Hyman,  1973,  p.5). 

5)  "...the  loss  of  interaction  that  results  in  class  discussion"  (Hyman,  1973,  p.5). 

He  goes  on  to  claim  that  in  order  to  complete  a  course  in  Individualized  Instruction,  a  student  must 
be  able  to  study  alone,  follow  directions,  remain  quiet,  listen,  and  follow  orders  (Hyman,  1973,  p.6).  If  he 
succeeds  he  is  quiet,  docile,  subordinate,  and  dependent.  Ironically,  these  are  some  of  the  issues  that 
individualization  is  supposed  to  address.  He  then  claims  that  schools  pre-adopt  children  to  the  industrial 
bureaucracy  by  preparing  them  to  work  alone  and  follow  orders  (Hyman,  1973,  p,  6). 

Unfortunately,  this  attack  was  too  broadly  based,  and  therefore  many  of  his  criticisms,  including 
the  loss  of  group  interaction  leading  to  less  peer  teaching,  when  applied  to  PSI  for  instance,  would  not 
hold.  For  in  the  Keller  setting,  one  of  the  main  methods  of  instruction  and  feedback  is  peer-tutoring. 

The  main  point  to  be  made  was  not  lost,  however.  He  prescribed  that  teachers  must  vary  their 
teaching  techniques  from  small  groups  to  one-on-one  work  to  large  lectures,  based  on  the  needs  of  the 
students. 
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Conclusion 


Individualized  Instruction  conies  in  many  forms,  from  Audio-Tutorials  to  Computer  Assisted 
Instruction.  There  are  many  common  themes  among  them.  Foremost  of  these  themes  is  the  effort  to 
improve  education,  The  principles  that  Individualized  Instruction  is  built  01 ,  regardless  of  specific  type,  are 
the  same  principles  that  all  of  education  is  built  on. 

Even  if  all  of  the  principles  that  are  part  of  the  theoretical  construct  of  one  of  these  types  of 
Instruction  are  sound  principles,  this  is  not  to  say  that  the  principles  of  the  Individualized  Instruction  are  all 
of  those  in  Education.  Because  not  alh)f  the  underlying  principles  of  Education  are  addressed,  group 
interaction  for  example.  Individualized  Instruction  has  its  limitation.  This  fact  must  be  realized  in  order  for 
Individualized  Instruction  to  be  improved. 

Each  type  of  instruction  addressed  the  effort  to  improve  with  a  different  set  of  prescription.  All 
have  been  heavily  criticized,  yet  that  is  to  be  expected.  As  Henry  Olds  (1985,  p.2)  put  it  "Most  schools 
are  still  unsure  about  acknowledging  the  reality  of  the  hand  calculator  as  a  tool..."  Change  is  a  slow 
process,  how  long  did  it  take  for  the  calculator  to  be  accepted  in  schools?  With  every  innovation  comes 
resistance  to  change,  and  rightfully  so.  It  would  be  foolish  to  incorporate  every  new  idea  that  came  into  the 
schools  within  a  year  of  its  inception.  If  something  cannot  withstand  criticism  then  it  should  not  be  a  part 
of  the  school  system. 

Individualized  instruction  is  still  a  relatively  recent  innovation,  and  will  remain  under  scrutiny 
until  several  criticisms  are  accounted  for.  Resistance  to  change  will  delay  the  implementation  of  these 
innovation,  even  after  their  limitations  are  accounted  for.  Individualized  Instruction  will  then  carry  out  its 
proper  role  in  the  improvement  of  education. 
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For  this  discussion,  technological  momentum  can  be  defined  as  the  increase  in  the  rate  of:  1 .  the 
evolution  of  technology,  2.  its  infusion  into  societal  tasks  and  recreations,  3.  society's  dependence  on 
technology,  and  4.  the  impact  of  technology  on  society. 

Some  may  not  like  the  use  of  the  word  evolution  in  relation  to  non-living  things.  Technology 
changes  due  to  environmental  stress,  responding  to  needs  or  wants  in  society.  Changes  in  existing 
technologies  try  to  alleviate  this  stress.  This  fits  the  concept  of  evolution  well,  so  that  word  will  be  used. 
Infusion  is  the  use  of  and  reliance  on  technologies  by  persons  in  the  course  of  completing  tasks  or  pursuing 
recreation  where  technology  was  not  used  before. 

The  use  of  the  word  technology  should  also  be  addressed.  Technology  can  be  looked  at  as  either  a 
process  or  a  product.  Although  the  first  may  be  considered  by  many  in  the  field  of  instructional  technology 
as  the  more  proper  way  of  thinking,  popular  usage  of  the  word  has  given  the  second  as  much  or  more 
weight.  When  technology  is  used  to  mean  only  a  product,  I  will  use  the  word  product  or  hardware,  which 
will  mean  both  hardware  and  software  for  the  remainder  of  this  paper, 

There  are  two  facets  to  technological  momentum.  The  first  is  too  much  technology.  The  second 
is  gaining  technology  too  fast,  without  fully  understanding  the  impact  that  the  technology  will  have.  There 
are  inherent  dangers  in  each  that  technologists,  instructional  technologists  in  particular,  need  to  take  into 
consideration  as  they  advance  technology's  influence. 

Why  do  criticisms  of  technology  exist?  Is  change  bad?  Is  less  work  for  the  teacher  bad?  This 
paper  is  an  expression  of  worry,  fear,  or  concern  dealing  with  a  perceived  flaw  in  something.  Sometimes 
the  criticism  is  constructive  and  meant  to  lead  to  the  correction  of  the  flaw.  Just  as  often,  the  flaw  is  used 
as  a  reason  to  disregard  or  discredit  the  thought  or  product  it  is  contained  within. 

Instructional  technology  has  been  talked  about  as  a  panacea  for  the  ills  of  education  and  society  in 
general.  Not  all,  however,  look  upon  instructional  technology  with  favor.  Many  are  opposed  to  the  use  of 
technology  within  education.  Others,  even  educational  technologists,  oppose  the  way  in  which  technology 
is  currently  used.  Several  of  these  criticisms  focus  on  the  idea  of  technological  momentum. 

It  is  often  easier  to  identify  problems  from  the  outside  of  an  issue.  People  not  intimately 
connected  with  an  issue  can  be  more  objective.  This  makes  it  important  for  insiders  to  listen  to  them. 
They  often  provide  insights  that  no  insider  would  generate. 

Instructional  technology  is  a  part  of  technology  (Heinich,  1984).  It  follows  from  this  that  the 
dangers  and  criticisms  of  technology  in  general  are  also  valid  for  educational  technology.  This  is  of  great 
concern  to  educational  technologists.  We  must  not  only  be  aware  of  what  we  are  doing,  but  also  what 
people  in  other  fields  and  disciplines  are  doing  as  well.  In  the  public's  mind,  all  technology  is  part  of  the 
same  machine.  All  parts  of  the  machine  bear  the  blame  when  one  part  fails  to  work  properly. 

The  fact  that  technology  is  gaining  momentum  is  very  easy  to  see  when  you  look  at  history.  The 
Stone  Age,  Bronze  Age,  and  Iron  Age  have  chronologically  dominated  humankind's  history.  Many 
significant  advances  were  made  during  these  periods.  However,  significant  advances,  such  as  the  wheel  and 
printing  press,  were  few  and  far  between.  Changes  in  thinking  or  design  were  slow  to  develop  and  just  as 
slow  to  spread. 

Not  so  during  the  Industrial  revolution.  Complex,  automated  machinery  began  to  be  used. 
Society  began  to  change  from  a  rural,  agrarian  base  to  an  urban,  industrial  base.  The  steam  engine  gave 
power  previously  undreamed  of,  making  possible  the  steam  locomotive  and  cotton  gin,  to  name  a  few.  The 
development  of  the  assembly  line  ushered  in  a  new  age  of  industrial  technique.  Change  followed  change, 
each  new  product  developing  further  or  leading  to  new  side-products. 

In  the  1970's,  aproximately  200  years  after  the  start  of  the  Industrial  Revolution,  the  world's 
dependence  on  oil  and  other  fossil  fuels  was  shown  when  OPEC  imposed  an  oil  embargo.  This  forced  the 
United  States  and  other  countries  to  look  at  their  use  of  oil-based  and  oil  using  products.  In  a  relatively 
short  period  of  time,  society  had  developed  and  become  dependent  on  such  things  as  the  internal  combustion 
engine  and  plastic. 

In  the  1950's,  the  first  super  computers  were  shown  to  the  world.  They  filled  entire  rooms  and 
could  follow  only  very  limited  instructions.  In  the  1970's,  a  scarce  20  years  after  their  introduction, 
computers  became  small  and  inexpensive  enough  for  personal  and  office  use.  In  1984,  Apple  released  a 
computer  that  used  a  new  idea,  the  graphical  user  interface  (GUI).  GUI  provided  a  visual,  intuitively 
understood  way  to  communicate  with  the  computer.  Now,  in  the  1990's,  nearly  every  college  faculty  and  a 
good  deal  of  the  population  at  large  has  a  computer  a  thousand  times  more  powerful  than  the  1950's 
versions  sitting  on  a  corner  of  their  desk.  We  are  joined  to  each  other  around  the  world  by  invisible 
communications  systems  that  can  provide  nearly  instantaneous  access  to  information  stored  almost 
anywhere  in  the  world.  We  are  now  in  an  Information  Age,  but  for  how  long? 
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Changes  have  become  exponential  as  time  goes  on.  Each  new  development  breeds  more  new 
ideas.  From  these  ideas  spring  forth  even  more  ideas.  On  and  on  it  goes,  and  whether  it  stops,  nobody 
knows.  Changes  in  early  technologies  were  simple,  often  being  a  change  in  material  or  design.  The 
Industrial  Revolution  saw  the  widespread-impact  of  a  different  type  of  technological  change,  a  change  in  the 
way  something  was  done.  Since  then,  the  rate  of  change  has  grown  exponentially.  Hardware  changes  much 
more  quickly  than  the  thinking  and  concepts  behind  them,  but  both  are  changing  rapidly  even  now.  New 
hardware  is  often  obsolete  within  five  years,  less  with  computers.  There  is  an  incredible  drive  for  change 
within  humankind.' 

Technology  is  often  associated  with  the  pursuit  of  progress,  doing  more  things  faster.  As  society 
changes,  its  needs  change.  The  changes  in  technology  are  often  responses  to  these  needs.  Sometimes  the 
new  technologies  or  products  lead  to  the  change  in  society's  needs  instead.  Technology  is  a  means  to  the 
end  of  progress.  However,  it  should  be  remembered  that  progress  is  not  always  good. 

The  development  of  the  cotton  gin  revitalized  a  crumbling  slave-based  economy  in  the  South.  In 
order  to  supply  the  world's  need  for  timber  and  beef,  we  are  cutting  down  enormous  tracts  of  rainforest  every 
day.  Entire  species  and  cultures  are  becoming  extinct  so  that  we  can  grab  a  Big  Mac©  at  the  drive-thru  on 
our  way  home  so  that  we  don't  miss  our  favorite  television  show. 

What  are  we  driving  home  in?  An  automobile  powered  by  gasoline.  What  is  gasoline  made  from? 
Gasoline  is  but  one  product  that  we  get  from  petroleum,  along  with  heating  oil,  rubber,  adhesives, 
solvents,  and  plastics.  In  our  need  to  keep  all  these  machines  going  and  make  more  of  them,  we  drill  into 
the  earth  in  more  and  more  places.  Each  drilling  site  upsets  the  natural  balance  of  the  immediate  area. 
When  oil  spills  occur,  it  upsets  the  balance  over  a  wide  area.  Even  more  hazardous  is  the  fact  that  oil  spills 
destroy  large  amounts  of  plankton,  the  small  ocean  plants  responsible  for  ninety  percent  of  the  earth's 
breathable  oxygen.  The  strip  mining  that  occurs  to  support  the  manufacture  of  products  and  the  chemicals 
we  make  further  damage  the  biosphere.  As  we  use  more  and  more  machines,  for  more  and  more  functions, 
these  atrocities  will  grow.  They  are  of  great  concern. 

Remember  that  technology  doesn't  just  respond  to  changes  in  society,  it  can  drive  those  changes  as 
well.  The  microwave  oven  is  one  of  those  products  that  has  changed  our  lives  greatly.  The  microwave 
oven  was  developed  in  response  to  the  needs  of  the  space  program.  It  was  only  one  very  small  part  of  the 
complex  system  required  to  put  humans  into  space.  Once  it  became  available  to  the  public,  however,  it 
became  a  focus  of  American  cooking.  No  longer  must  we  wait  for  our  food.  Preparation  time  has  been 
drastically  reduced,  freeing  time  for  other  pursuits.  Individual  portions  are  more  easily  prepared,  making 
large  family  meals  a  less  frequent  occurrence.  Society's  need  for  speed  is  spreading  into  all  parts  of  our  life. 
How  long  until  it  reaches  our  educational  system? 

There  have  been  changes  in  our  chosen  forms  of  entertainment,  too.  Radio  was  the  first  non- 
written  entertainment  to  enter  people's  homes  on  a  wide  scale.  For  the  first  time,  large  numbers  of  people 
could  be  reached  at  the  same  time  over  great  distances.  This  was  even  more  true  of  television,  when  it  was 
introduced.  At  first  people  only  had  to  choose  between  one  or  two  radio  or  television  stations.  As  their 
popularities  grew,  there  were  more  and  more  stations.  This  meant  more  and  more  choices.  Now,  with 
cable,  it  is  not  unusual  for  there  to  be  a  choice  between  30  or  40  channels,  each  with  something  slightly 
different,  each  competing  for  people's  interest. 

This  competition  for  interest  has  become  a  race  of  who  can  have  the  funniest,  saddest,  most  real, 
most  exciting,  or  even  bloodiest  programming.  To  keep  our  attention,  plots  change  rapidly.  This  has 
caused  a  noticeable  drop  in  attention  spans  in  students  who  are  part  of  the  "MTV  Generation."  Surely  an 
unplanned  for  side-affect,  but  still  a  result  of  the  rapid  change  in  television. 

Criticisms  of  technology  have  been  around  for  a  very  long  time.  Mary  Shelly's  Frankenstein 
addresses  the  issue  of  humans  losing  control  over  their  creations.  Dr.  Frankenstein  seeks  control  over  life 
and  death  itself,  he  wishes  to  create  life.  He  is  even  successful,  bringing  to  life  a  conglomeration  of  pieces 
from  corpses  recovered  from  the  graveyard.  Having  found  the  secret  to  giving  life  to  a  being,  Dr. 
Frankenstein  only  realized  that  he  could  do  such  a  thing.  He  did  not  stop  to  think  whether  he  should  instill 
life  where  there  is  none. 

These  questions  remain  today,  even  with  the  above  example.  Cloning  is  a  cutting  edge  field. 
Even  today,  court  cases  rage  over  the  question  of  artificially  prolonging  a  person's  life  through  the  use  of 
machines  and  drugs.  What  medicine  can  do  today  would  seem  almost  as  wondrous  as  creating  life  to 
someone  from  just  250  years  ago.  In  some  instances  it  can  be  a  very  helpful  thing.  In  others,  it  is 
unwanted  and  is  hated.  Many  people  believe  that  not  everything  can,  or  should,  be  controlled.  This  may  be 
even  more  true  in  education  than  in  other  parts  of  society. 

Control  is  a  necessary  part  of  technology  and  instructional  technology.  Control  is  needed  over 
outcomes,  procedures  and  evaluation.  Without  this  control  to  define,  the  instructional  technologist  will  fail 
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in  designing  an  effective  system.  It  is  only  when  variables  are  eliminated  or  successfully  manipulated  that 
control  can  exist  Only  when  control  exists  can  instructional  technologists  design  an  efficient  system. 
However  control  is  something  we  are  taking  away  from  the  teachers  and  technologists.  It  is  being  given  to 
the  students  in  the  hopes  of  involving  them  in  their  own  education,  individualizing  each  student's  education, 
and  making  each  student's  education  relevant  to  their  life.  Without  control  over  these  things,  are 
technologists  going  to  be  able  to  plan  instruction  usable  by  everyone? 

This  need  for  control  is  a  limiting  factor  for  instructional  technologists.  Certain  subjects,  like 
mathematics  and  physical  sciences,  lend  themselves  to  the  rational,  logical  organization  that  is  the  hallmark 
of  instructional  design.  Not  all  subjects  or  outcomes  fit  in  so  nicely,  however.  It  is  very  hard,  perhaps 
impossible  to  put  attitudes,  creativity,  and  understandings  into  behavioral  objectives,  another  hallmark  ot 
instructional  design.  A  problem  of  instructional  technology  is  that  objectives  are  often  changed  when 
instruction  does  not  lead  to  the  desired  behavioral  changes  (Merrill,  1971). 

we  adjust  what  we  teach  to  what  we  can  teach  in  a  certain  way,  rather  than  adjusting 
our  teaching  to  the  human  being.  Though  we  all  look  forward  to  the  day  of  more  use  of 
educational  technology  in  freeing  the  teacher  from  tutoring,  reviewing,  drilling,  and 
evaluating,  we  must  also  recognize  the  danger  of  de-emphasizing  the  development  of 
children's  imaginations,  of  creativity.  (Driscoll,  1978) 

This  passage  both  praises  instructional  technology  and  warns  against  its  use.  The  benefits  of  increasing 
technology  in  the  schools  are  many.  But  will  we  limit  our  instruction  to  only  those  things  we  can  control 

and  defineMost  educatiopa!  technologists  don't  design  systems  or  instruction  thai  they  don't  want  used. 
These  systems  are  used  by  people.  The  systems  involve  people.  Therefore,  the  concerns  that  people  have 
with  technology  and  instructional  technology  in  particular  should  be  taken  into  consideration.  As  peoples 
concerns  with  technology  grow,  so  will  their  dissatisfaction  with  the  educational  system. 

One  of  the  major  criticisms  of  instructional  technology  is  dehumanization.  There  are  certain 
characteristics  that  are  regarded  as  being  "human."  Among  these  are  creativity,  feelings,  compassion, 
individualism,  passion,  and  understanding.  When  something  other  than  a  human  possesses  one  or  more  ot 
these  traits  it  is  often  personalized  or  given  credit  for  being  like  humans.  If  a  person  shows  a  lack  of  these 
traits  they  are  said  to  be  mechanistic,  cold,  or  inhuman.  There  is  a  great  deal  of  importance  given  to  these 
traits'  and  anything  that  hinders  or  retards  their  development  or  expression  is  considered  undesirable. 

'     One  of  the  greatest  of  the  dehumanizing  arguments  is  that  technology  seeks  to  treat  every  person 
the  same  way  disregarding  individual  attributes,  abilities,  feelings,  and  needs.  Individual  needs  are  only  met 
when  they  meet  the  demands  of  the  system  (Streibel  1986).  As  the  use  of  technology  in  education 
increases,  there  is  the  danger  that  students  will  be  treated  less  as  the  individuals  they  are  and  more  like 

interchangeable  parts.  .       .    ,  . 

The  students  lose  their  individuality.  They  are  treated  as  another  variable  in  the  system,  lnis 
student-as-product  ideology  has  been  a  major  criticism  of  instructional  technology.  The  concept  of  people 
as  parts  is  offensive  to  many  educators.  This  can  make  instructional  technology  very  unpopular  to  the 

average  educator.  ...  ■  tt.„ 

Educators  are  also  in  danger  of  becoming  interchangeable  parts  in  the  instructional  system.  The 
purpose  of  instructional  design  is  the  identification  and  manipulation  of  individual  variables  in  the 
instructional  process.  To  many  instructional  designers,  teachers  are  just  another  variable.  Individual  teacher 
characteristics  are  disregarded.  One  teacher  becomes  just  as  good  or  as  bad  as  any  other  teacher.  The  craft, 
or  art  if  you  prefer,  of  teaching  will  be  replaced  by  the  competency  of  teaching. 

Bates  writes  of  a  cult  of  competency.  It  is  described  as  "one  current  mechanism  through  which 
certain  changes  are  being  advocated  in  the  professions  and  their  associated  education  programs.   The  cult  of 
competency  is  not  an  educational  movement,  it  is  a  managerial  movement  with  the  goal  of  increasing 
efficiency  between  differing  sections  of  society.  The  effect  of  this  movement  will  be  the  deterioration  of 
teachers' skills.  (Bates,  1992) 

The  overall  result  of  the  cult  of  competency,  paradoxically,  is  likely  to  be  a  system  that 
may  well  serve  to  restrict  the  development  of  expert  professionalism,  ...In  constraining 
the  open-ended  nature  of  professional  activity  at  this  expert  end  of  the  scale  we  may  well 
produce  a  competency  based  system  in  which  substantially  more  professionals 
performance  will  be  reduced  to  the  level  of  basic  skills.  We  may  well  end  up  with  the 
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very  thing  we  do  not  want:  professionals  in  education  as  elsewhere  who  are  indeed  barely 
competent,  (p  7) 

If  teachers  and  students  are  reduced  to  interchangeable  parts,  another  phenomenon  will  occur.  The 
skills  required,  both  by  teachers  and  students,  will  be  reduced  to  the  barest  technical  levels. 

As  teachers  are  encouraged  to  adopt  new  technology,  they  will  also  have  to  adopt  new  roles  and 
duties.  Teachers  will  become  interchangeable  since  their  skills  will  be  at  a  minimal  mechanical 
competency.  (Koetting,  1988) 

As  the  teaching  profession  has  become  an  increasingly  highly  skilled  technology 
with  a  primary  emphasis  on  methods  and  outcomes,  teachers  have  been  rewarded  for 
guiding  their  practice  in  ways  amenable  to  their  technology.  As  Macdonald  suggests 
(1975),  this  notion  implies  that  "teachers  are  potentially  interchangeable,"  and  leads  to 
viewing  productive  activity  as  something  learned  and  performed  "mechanistically."  Thus, 
any  "good"  teaching  activity  can  be  reproduced  by  any  other  teacher,  and  "...all  productive 
teaching  is  measurable  in  terms  of  the  criteria  of  the  accountability  in  use"(pg446) 


Diversity  is  an  important  idea  today.  Awareness  of  and  appreciation  for  the  differences  among 
people  is  a  focus  of  many  educational  reforms.  It  is  the  current  theme  for  Kappa  Delta  Pi,  an  international 
educational  organization.  Tolerance  for  differences  is  not  only  a  goal  for  our  students,  but  for  today's 
teachers  as  well.  The  accommodation  of  learning  style  and  cultural  differences  is  increasing  its  importance 
as  an  issue  in  teacher  preparation  programs  and  continuing  teacher  education.  A  result  of  this  effort  is  the 
formation  of  a  single  global  community.  The  idea  of  a  diverse  community  is  at  odds  with  a  technological 
approach  to  education.  The  conformity  that  is  forced  by  designed  instruction  works  to  inhibit  individual 
differences. 

Coupled  with  this  faster  rate  of  societal  change  will  be  an  increasing  homogenization  of 
different  nation.5,  and  cultures.  As  the  centralization  of  production  grows,  instructional 
units  will  become  ever  more  standardized;  whatever  cultural  differentiation  takes  place  in 
programming  to  adapt  to  different  audiences  may  be  as  much  'window  dressing'  as  actual. 
This  growing  uniformity  will  speed  the  rate  of  global  change  and  facili';ate  intercultural 
communication,  but  will  also  reduce  the  pool  of  diversity  and  pluralism  on  which  the 
human  race  can  draw.  (Dede,  1981) 

Technological  advance  is  based  on  control  and  uniformity  and  being  able  to  predict  outcomes. 
Even  as  we  begin  io  preach  about  the  community  and  diversity,  we  advance  the  technology  that  may  very 
well  destroy  these  traits  within  us. 

Adapting  to  new  technology  will  become  easier  as  time  goes  on.  Speaking  of  a  popular  piece  of 
technology,  McPherson  writes,  "The  more  adaptable  computers  become  to  the  requirements  of  human 
beings,  the  less  human  beings  will  have  to  adapt  to  the  requirements  of  computers"  (McPherson,  1984). 
The  Macintosh's  graphical  user  interface  made  it  much  easier  to  use  computers.  You  no  longer  needed  to 
know  commands  and  codes  that  sounded  and  looked  like  Greek.  This  ease  of  use  resulted  in  an  increased  use 
of  computers  in  many  areas:  home,  business,  and  school.  Now,  hardware  and  software  exist  that  can 
recognize  an  individual's  handwriting  and  convert  it  to  type.  The  addition  of  speech  recognition  and  vocal 
control  to  computers  will  make  computers  even  easier  to  operate.  Soon,  we  will  be  able  to  talk  to  our 
computers  on  nearly  the  same  level  as  another  person. 

The  fact  that  technology  is  becoming  easier  to  use  leads  to  its  increased  use,  whether  it  is  needed  or 
not.  "A  common,  and  often  justifiable,  criticism  of  instructional  design  is  that  it  results  in  doing  better 
what  shouldn't  be  done  in  the  first  place"  (Gustafson,  1971 ).  Again,  just  because  something  can  be  done, 
does  not  mean  it  should  be  done  or  will  have  positive  results. 

The  preoccupation  of  instructional  technology  with  management  theory  and  efficiency  are  often 
seen  as  weaknesses.  It  is  not  that  educators  are  against  efficient  instruction,  it  is  the  degree  and  manner  in 
which  instructional  technology  uses  these  ideas. 

The  managerial  approach  of  instructional  technologists  includes  the  students-  as-products  idea.  This 
idea  is  disliked  by  many  educators.  The  idea  that  students  are  components  that  move  through  the 
educational  system,  each  being  treated  in  the  same  manner,  doesn't  coincide  with  the  view  of  teaching  held 
by  many  educators.  For  many,  teaching  is  an  interactive  process.  This  interaction  is  primary  social 
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interaction  between  individuals  or  groups,  i.e.  other  people.  While  instructional  technology  strives  for 
interaction,  it  is  interaction  between  people  and  machines.  Replacing  people  with  machinery  is  a  very 
dangerous  proposition.  It  is,  however,  the  direction  instructional  technology  is  heading. 

The  increased  use  of  managerial  models  in  instructional  technology  relates  to  the  control  issue  as 
well.  As  more  technology  is  used  in  the  classroom,  educators  at  the  local  levels  lose  more  and  more 
control  over  what  is  taught  and  the  method  of  instruction. 

The  control  of  instruction  will  mirror  the  change  in  control  of  production.  (Bates,  1986) 

...the  development  of  industrial  capitalism  was  based  not  so  much  on  technical  advances 
in  production  methods,  but  rather  upon  alterations  in  the  management  practices  which 
removed  control  of  the  production  process  from  the  hand  of  artisans  and  relocated  it  in  the 
hands  of  owners.  Simultaneous  with  this  process  was  the  deskilling  of  the  workers,  (pp 
4,  5) 

Again,  the  history  of  general  technology  can  be  used  to  foretell  the  future  of  educational 
technology.  Just  as  control  of  production  was  taken  from  the  workers,  so  too  will  the  control  of 
instruction  be  taken  from  teachers.  The  control  will  rest  with  the  instructional  designer,  who  is  often  far 
removed  from  the  classroom  environment. 

There  is  also  the  change  of  roles  for  educators.  Educators  will  stop  being  deliverers  of  instruction 
and  become  managers  of  instruction.  (Bates,  1986) 

What  teachers  may  well  be  experiencing. ..is  an  intensification  of  pressure  to 
conform  with  particular  instructional  models  which  are  determined  by  researchers  as  "more 
effective."  Principals  and  Instructional  Leaders  of  various  kinds  are  likely  to  be  subject  to 
increasing  barrages  of  information  on  "successful"  instructional  techniques  and  be  urged  to 
ensure  their  adoption  by  teachers.  Systems  of  sanctions  and  rewards  (incentive  systems) 
will  be  introduced  to  reinforce  the  adoption  of  the  new  techniques  and  to  engineer  a 
political  apparatus  of  regulation  which  will  parallel  the  particular  organization  of  the 
tasks  of  educational  production  specified  by  the  experts. 

Such  developments  are  likely  to  lead  both  to  an  increasing  emphasis  on  the  logic 
of  bureaucratic  rationality  (Rizvi,  1986)  and  to  further  restrictions  on  the  responsible 
autonomy  of  the  teaching  profession.  It  may  well  lead  to  further  development  of  what 
Webb  (1985)  calls  status  panic  among  teachers.  Moreover,  such  developments  are 
certain,  in  my  view,  to  further  develop  a  technical  notion  of  educational  practice  which  is 
devoted  to  managerial  rather  than  educational  ends,  (pp  1 1,12) 

Again,  the  switch  to  managing  instruction  eliminates  much,  if  not  all,  of  the  interpersonal 
relationships  that  educators  take  pride  in.  Part  of  teaching  has  always  been  the  challenge  of  communicating 
with  the  students,  passing  knowledge,  skills,  and  thoughts  to  a  new  generation.  Instructional  technologists 
would  reduce  the  complex  interpersonal  interactions  to  a  cold,  sterile  overseeing  of  instructional  tasks.  Are 
we  promoting  an  educational  system  where  the  terminal  degree  is  an  MBA,  not  an  MSEd? 

The  fact  is,  technology  is  coming  on  too  fast.  Educators  can't  keep  up  with  the  rapid  change 
taking  place.  Most  schools  don't  have  the  knowledge  necessary  to  integrate  technology  effectively  into 
their  system.  How  often,  even  in  higher  education,  do  we  see  brand  new  equipment  sitting  in  an  office  or  a 
lab,  unused?  Often,  people  simply  don't  know  how  to  use  the  new  products  or  methods.  The  tremendous 
advances  in  technological  capability  has  not  been  paralleled  by  advances  in  training  or  ability  at  the  level  of 
our  schools.  Often,  even  our  teacher  training  programs  lack  such  instruction. 

Schools  often  find  themselves  at  the  mercy  of  distributors  and  companies  who  want  to  make  a 
profit.  Administrators,  School  Boards,  and  teachers  often  purchase  instructional  materials  or  hardware  that 
never  get  used.  When  methods  or  hardware  that  don't  work  are  acquired,  what  is  the  solution?  Apply  more 
technology,  of  course.  (Nichols,  1987)  If  a  product  or  method  doesn't  work,  it  can  be  made  to  work  if 
enough  time  and  money  is  spent  on  making  it  work.  The  idea  that  another  product,  or  even  no  product, 
could  do  as  good  or  even  a  better  job  infrequently  comes  to  mind. 

Technology  often  breeds  the  need  for  more  technology.  Usually,  the  new  technology  is  designed 
to  overcome  problems  of  or  solve  problems  created  by  older  technology.  There  is  always  the  drive  to  do 
more,  to  do  it  faster,  and  to  do  it  less  expensively.  This  is  a  very  dangerous  idea.  Take  for  example  the 
technology  of  war.  Each  invention  was  never  good  enough  at  killing.  Sticks  and  stones  became  swords 
and  bows.  Then  guns  and  cannon  were  developed.  Next  came  explosive  shells,  then  missiles.  Finally,  the 
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nuclear  bomb  was  developed.  Were  designers  happy?  No,  they  wanted  even  more  destructive  power. 
Where  does  it  stop?  Now  think  about  this  trend  in  relation  to  education,  where  will  the  instructional 
designers  stop?  Will  all  behaviors  and  actions  be  reduced  to  pat  little  formulas?  Will  every  aspect  of  life 
become  an  equation? 

The  concerns  stated  in  this  paper  are  just  that,  concerns.  They  are  not  condemnations  of 
technology.  Technology  has  many  advantages  and  is  here  to  stay.  We  will  not  get  rid  of  technology. 
Nothing  we  do  will  stop  the  evolution  of  the  processes  and  products  that  are  becoming  integral  parts  of  our 
society.  Even  if  we  could  stop  technological  momentum,  it  would  not  necessarily  be  a  good  thing. 

There  is,  and  will  always  be  that  within  humankind  that  craves  and  needs  change.  There  is  also 
that  within  humankind  that  fears  change  and  will  try  to  stop  it.  Envision  the  process  of  change  as  a  river. 
The  fear  that  inhibits  change  is  a  dam  across  that  river.  The  pressure  of  the  river  builds  and  builds  behind 
the  dam.  Finally,  the  dam  breaks  and  there  is  a  massive  rush  of  water.   Only  for  us,  it  is  change.  This 
power  can  be  harnessed  and  used  for  productive  purposes,  just  as  a  controlled  flow  through  a  dam  is  used  to 
create  energy.  If  it  is  held  back  for  too  long,  though,  it  will  eventually  burst  through,  carrying  us  along 
whether  we  are  ready  for  it  or  not.  ° 

We  should  try  to  direct  and  harness  the  power  imbedded  within  technological  momentum.  The 
benefits  that  can  be  gained  from  technology  are  many.  However,  uncontrolled  advancement  will  cause  more 
harm  than  good.  We  need  to  listen  to  the  concerns  brought  to  us  by  the  public,  by  educators,  and  by  our 
peers.  Let  us  address  the  issues  facing  us.  Let  us  become  ethical,  responsible  practitioners  of  instructional 
technology. 
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Almost  since  its  inception,  the  work  "dehumanization"  has  caused  many  to  feel  apprehensiveness, 
especially  as  the  words  relate  to  Educational  Technology.  Like  "human  engineering",  dehumanization 
carries  a  stigma  of  Orwellian  proportions  which  is  not  likely  to  dissipate.  This  phenomenon,  as  it  relates 
to  the  field  can  be  traced  back  to  the  turn  of  the  century,  or  possibly  even  further.  We  have  witnessed  a 
revolution  in  classrooms  since  those  days,  from  one  room  schoolhouses  to  giant  city  and  metropolitan 
school  districts.  "The  building  of  technologies  have  made  it  possible  for  people  to  cluster  together  in  huge 
numbers  in  urban  centers  where  they  live  a  segregated  kind  of  life  which  no  one  notices-not  segregated 
according  to  black  and  white,  but  segregated  according  to  income  bracket"  (Taylor,  381). 

Some  would  argue  that  the  implementation  of  new  technologies  is  recent  years  has  caused  us  to 
forget  our  own  humanity.  "What  is  good  must  have  accountability,  must  have  behavioral  objectives;  in 
effect  must  be  efficient"  (Meierdiercks,  8-9).  This  paper  is  a  brief  analysis  of  Educational  Technology's 
critics  from  the  late  1950's  through  the  present  time.  It  also  serves  as  a  brief  study  of  how  their  rhetoric  has 
affected  the  structure  of  the  K-12  classroom  in  recent  years. 

"The  emerging  era  of  high  technology  has  made  its  promises  that  life  will  be  easier  and  its 
problems  more  easily  solved.  The  result  of  this  slow  but  persistent  change  has  been  a  shrinking  concept  of 
education  as  we  have  looked  toward  a  concept  of  "training"  rather  than  education"  (Nelson,  9).  When  looked 
at  in  retrospect,  it  is  truly  astounding  how  much  technology  has  influenced  each  of  our  lives.  More 
innovations  have  been  developed  in  the  last  58  years  than  in  most  of  recorded  history,  "as  time  passes, 
technology  multiplies  in  staggering  exponential  terms"  (Smith,  3). 

The  fundamental  problem  addressed  in  this  paper,  however,  does  not  concern  itself  with 
technology's  rapid  growth  exclusively.  This  is  mainly  because,  its  growth,  for  all  intents  and  purposes,  is 
simply  a  factor,  capable  of  neither  good  nor  bad.  It  is  the  importance  we  attribute  to  certain  things, 
however,  which  gives  them  their  intrinsic  value  in  our  daily  lives.  Technology,  then,  as  it  pertains  to 
education,  is  certainly  no  exception.  According  to  many  critics,  the  variable  of  technology  which  once 
seemed  so  promising,  has  fallen  from  use  into  abuse  in  recent  times.  It  is  this  abuse,  they  contend,  which 
has  served  as  a  catalyst  in  aiding  dehumanization  in  the  K-12  classroom  setting. 

The  Sixties:     Revisionism,  Objectivity  and  the  Teaching  W  ine 

The  decade  of  the  sixties  opened  with  a  continued  emphasis  on  hard  sciences  and  objectivity  in  the 
American  curriculum.  While  records,  tapes,  and  film  continued  to  gain  popularity  in  classrooms,  many 
teachers  struggled  to  maintain  a  sense  of  individuality  both  for  themselves  and  for  their  students.  Some  of 
the  era's  new  inventions,  which  would  now  be  considered  novelties,  posed  a  valid  threat  to  teachers  who 
were  uncomfortable  with  technology's  implementation  in  the  school  environment.  One  such  innovation 
was  a  clever  little  devise  created  in  1962  by  Kobler  and  Moore.  It  was  know  as  the  talking  typewriter,  and 
although  it  was  quickly  phased  out  as  a  passing  fad,  many  students  took  a  quick  liking  to  it. 

Hnrold  Taylor,  in  his  1964  address  to  the  DA VI  Convention  stated  that  technology  has  often  been 
a  key  factor  in  imbalancing  our  values  as  a  nation.  In  essence,  Taylor  surmised  that  America  had  become  "a 
society  which  is  as  dazzled  by  technological  accomplishments  as  it  is  powerless  to  act  on  the  most  serious 
problems  created  by  them"  (Taylor,  382).  During  that  time  period,  it  appears  that  many  of  Taylor's 
comments  were  not  terribly  far  from  the  truth  of  the  situation.  He  saw  no  needs  or  incentives  for  students 
to  stay  in  schools  that  were  quickly  becoming  (as  Herbert  Kohl  once  defined  them),  "grim,  joyless  places." 

Taylor  was  not  the  only  critic  who  harbored  this  revisionist  attitude  toward  mod  rn  education.  In 
later  years,  other  personalities  emerged  to  stress  the  need  for  increased  humanity  in  the  system.  "One  might 
add  that  the  future  would  take  on  profoundly  different  meaning  it  ilie  revisionist's  vision  were  to  prevail." 
Bowers  was  also  quick  to  not  the  "symbolic  fragmentation  and  increasing  loneliness  that  appears  to  result 
from  the  low-context  form  of  modern,  technologically-based  cultures"  (Gowers,  280). 

Nevertheless,  many  in  education  continued  to  look  at  technology  through  rose-colored  glasses,  and 
the  many  education  possibilities  it  seemed  to  promote.,  Interest  was  even  re-kindled  in  old  educational 
devices,  some  of  which  had  been  operationally  dormant  for  30  or  more  years.  One  such  instrument  was 
Pressey's  teaching  machine,  which  distributed  rewards  (such  as  candy)  for  correct  answers  to  questions.  This 
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renewed  excitement  in  objective  theories  helped  to  put  psychologists  "like  B.  F.  Skinner  back  in  the 
education  spotlight. 

His  1968  book,  "Teaching  Machines",  chronicled  his  use  of  conditioning  upon  common  pigeons, 
and  the  results  he  received.  Though  it  was  simply  a  collection  of  his  old  essays,  its  wild  popularity  seemed 
to  indicate  that  schools  were  not  ready,  or  particularly  fond  of  moving  away  from  behavioral  objectives. 

The  Seventies:     CAI,  Educational  Television,  and  Resistance 

The  seventies  became  a  witness  to  large  scale  computer  experiments,  CAI,  reading  instruction, 
cable  television,  and  intensified  individual  instruction.  Readi  Television,  was,  by  this  theme,  a  powerful 
and  well-integrated  tool  for  both  elementary  and  secondary  education.  Shows  like  Sesame  Street  and  the 
Electric  Company  became  popular  in  the  curriculum  for  students  around  the  ages  of  7  through  10.  Films, 
filmstrips,  '.ape  recorders,  and  phonographs  became  a  mainstay  in  seventies  classrooms.  Though  the 
personal  computer  was  still  in  development  in  the  early  seventies,  the  excitement  had  already  begun  over 
their  evsntual  implementation. 

In  1972,  Nila  Banton  Smith  cited  three  fundamental  problems  with  technology  in  regard  to  its  use 
in  education:  Cost  (4  to  6  million  at  the  time),  resistance,  and  most  notably,  its  dehumanizing  effect.  "A 
controlled  study  recently  made  with  junior  high  school  students  at  Stanford  University  indicates  that 
computers  are  more  charismatic  then  teachers"  (Smith,  12).  Though  Smith  cited  this  as  a  potential 
problem,  she  also  indicated  that  it  was  perhaps  better  for  the  students  to  make  a  "truce"  with  technology 
rather  than  resist  it.  This  "truce"  could  generally  be  thought  of  as  a  compromise  between  the  learner  and  the 
inevitable  advent  of  the  impressive  machine. 

Eric  Hoffer  was  once  quoted  as  saying  that  "We  never  can  be  fully  prepared  for  that  which  is  new. 
We  need  to  adjust  ourselves,  and  every  radical  adjustment  is  a  crisis  in  self-esteem. ..it  needs  inordinate  self 
confidence  to  face  drastic  change  without  inner  trembling."  But  for  many  educators  d    ing  the  seventies,  the 
inner  trembling  had  already  begun.  For  some,  the  threat  of  new  technology  only  served  to  compound  their 
struggle  to  teach  a  given  curriculum.  "When  technology  is  added  to  the  traditional  classroom,  in  many 
cases,  the  addition  is  simply  another  overlay  of  administrative,  managerial  detail"  (Evertson,  Stallings,  58). 

John  Belland  also  took  note  of  this  phenomenon  examined  some  of  its  inherent  problems  in  his 
1975  essay, "  1984  is  Only  Nine  Years  Away:  Will  School  Media  Programs  Humanize  or  Dehumanize 
Schooling?"  again,  images  of  Orwell  are  conjured  up  for  the  reader's  consumption,  as  Belland  confronts  the 
man-machine  controversy  directly.  In  his  essay,  the  author  makes  it  quite  evident  that  the  choice  of  whether 
to  humanize  or  dehumanize  rests  solely  with  the  human  race.  "We  are  faced  with  the  very  real  question  of 
whether  the  technology  which  has  been  evolved  by  human  beings  to  meet  human  needs  serves  to  liberate  us 
or  serves  to  enslave  us"  (Belland,  3).  Furthermore,  Belland  contends  that  media  are  simply  extensions  of 
people,  and  that  technology  should  be  policed  to  insure  that  they  remain  that  way. 

Systems  Anulysis  and  Individualized  Instruction-Continued 

Systems  analysis  was  one  of  the  many  controversial  trends  that  surfaced  during  the  1970s. 
Though  the  sixties  saw  much  of  the  advent  of  this  instructional  technique,  the  seventies  helped  to  s;  turate 
the  process  and  make  it  an  integral  part  of  the  overall  curriculum.  At  the  time,  it  may  have  seemed  that  the 
term  "individualized  instruction"  implied  a  more  student-centered  approach.  Some  critics,  however,  regarded 
it  as  nothing  more  than  the  complete  antitheses  of  its  name. 

In  any  case,  such  a  concept  of  individualizing  hardly  meets  the  demand  to  personalize  the 
school,  for  critics  arc  aiming  their  attacks  at  the  content  taught  as  well  as  the  procedure 
for  teaching  it.  Obsolete  or  irrelevant  material  learned  in  small  segments  is  no  less 
obsolete  or  irrelevant.  Segmented  junk  is  still  junk  (Hyman,  414). 

In  addition,  Hyman  concludes  that  individualized  instruction  contributed  to  loss  of  classroom 
interaction,  discussion,  peer  conferencing,  and  other  related  schoolroom  activities,  The  focus,  according  to 
Hyman's  reasoning,  should  be  on  content  rather  than  procedure.  This  sentiment  was  shared  by  other  critics 
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of  the  period,  including  John  W.  Alden.  It  was  Alden's  argument  that  educators  of  the  seventies  were 
unable  to  take  technology  and  apply  it  in  a  way  which  was  beneficial  in  promoting  learning  among 
students.  "I  believe  that  we  are  so  enamored  with  techniques,  particularly  when  they  are  computer  related, 
that  we  often  lose  sight  of  the  reason  for  the  technique  in  the  first  place"  (Alden,  3). 

Many  agreed  with  Alden's  views  during  the  seventies,  and  nofd  that  systems  analysis  had  not  its  focus  as 
an  interlocking  program.  The  human  element  of  the  process  was  slipping  away,  with  technological 
momentum  gradually  taking  its  place.  Overall,  the  general  attitude  persisted  that  educational  institutions 
were  producing  students  who  were  merely  meek  repeating  m;  hines.  "Most  educators  would  not  like  to 
hear  students  described  as  meek;  but  received,  memorizing,  and  repeating  would  hardly  raise  an  eyebrow" 
(Christenbury,5). 

This  submission  of  man  to  machine  was  something  that  continued  to  both  fascinate  and  disturb 
critics  during  the  seventies.  Kenneth  D.  Benne,  in  a  1975  article,  outlined  some  of  the  dehumanizing  trends 
he  saw  inherent  in  the  modern  day  educational  system.  "Schooling  is  seen  and  practiced  as  a  process  in 
which  "correct"  environmental  influences,  predetermined  by  adults,  are  brought  to  bear  upon  assuredly 
-passive  and  plastic  learners  to  produce  educational  "pn">  J-.icts"  needed  by  society  (Benne,  45). 

Unlike  more  conventional-minded  educational  technology  critics  of  his  time,  Benne  took  a  more  creative 
approach  to  analyzing  the  situation  by  v.. iting  a  poem.  In  it,  he  personifies  technology  as  an 
overwhelming  deity  which  had  awestruck  it  followers  to  the  point  of  nearly  complete  submission: 

Faceless  or  whith  averted  faces,  I  stood  alone  and  I  could  hear 
Their  almost  human  voices-impressive  sound 
Well  amplified,  most  high-fedility-commanding  "Kneel!" 
I  did  not  kneel.  And  from  me  came  a  bleat- 
Most  poorly  modulated  low-fidelity-  "I  do  not  feel  your  right  to  make  me  kneel. 


This  deity,  by  the  end  of  1970'2,  was  certainly  a  force  to  be  reckoned  with,  especially  in  regard  to 
teaching.  The  teacher's  and  was  continuing  its  demise.  "Teaching  related  issues  appear  not  to  be  taken  very 
seriously;  issues  such  as  individual  philosophical  orientations  and  their  reasons  for  being  a  teacher" 
(Jorgenson,  264). 

Ken  Meierdiercks,  in  a  1980  article,  noted  technology's  ability  to  swallow  virtually  anything  that 
stands  in  its  was.  "Our  technological  society  is  much  more  dynamic  and  devouring  than  any  other  in 
history  (Meierdiercks  8,9).  By  this  time,  society  had,  in  many  ways,  become  the  technological  future  world 
which  George  Orwell  had  boldly  envisioned. 


The  Eighties:     Militarization,  Computers,  Vocabulary,  and  Science 

Militarization 

The  repercussive  embers  of  Sputnik  remained  evident  throughout  the  eighties,  as  critics  continued 
to  evaluate  how  well  the  educational  movement's  long  term  goals  were  met.  During  the  'Reagan 
Administration,  there  was  an  attempt  to  hold  military  funding  to  a  more  militaristic  standard.  "In  order  to 
improve  instruction  and  in  order  to  legitimate  their  profession  through  the  incorporation  of  new 
technologies  and  the  latest  applications  of  scientific  research,  the  schools  have  unwittingly  welcomed  the 
Trojan  Horse  of  military  prerogatives  within  their  gates"  (Nichols,  182). 

The  teacher,  in  a  sense,  is  regarded  as  the  unassuming  victim  of  this  abrupt  technological  invasion. 
The  "microcomputer  phenomenon"  which  began  in  the  eighties  and  continues  to  this  day  is  viewed  by 
Noble  as  the  "most  visible"  symbol  of  educational  dependence.  In  other  words,  the  very  military  which 
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helped  develop  computer  science  was  now  helping  to  dehumanize  the  educational  school  process.  "To  the 
degree  that  our  purchases  are  less  professionally  conceived  activity  attending  primarily  to  education,  and 
more  a  social  imperative  connected  to  economic,  political  or  military  gain,  our  buying  hardware  can  be 
called  negative"  (Nichols,  1 28). 

Meanwhile,  teachers  continued  to  feel  their  influence  waning  as  they  too  became  slaves  to  the 
powerful  forces  of  Educational  Technology.,  This  problem,  as  it  relates  to  the  study  of  language  arts  was 
recognized  and  commented  on  by  J.  Amos  Hatch  in  1984.  "Teachers  look  and  act  like  instructor-technicians 
as  they  teach  reading.  They  march  their  students  along  the  reading  system  and  instructional  management 
assembly  lines  using  timetables,  materials,  and  strategies  p described  by  invisible  decision  makers"  (Hatch, 
248). 

Many  critics  found  this  trend  particularly  unnerving  because  it  tended  to  discount  the  very  art  of 
teaching  itself.  Individual  teaching  styles,  as  well  as  knowledge  of  pedagogy  became  incidental  when 
compared  directly  with  facilitation  skills.  Part  of  this  downward  spiral,  according  to  Hatch  and  others,  could 
be  attributed  to  Educational  Technology. 

Kalhryn  Nelson,  in  her  1985  paper,  "Liberal  Arts  in  a  High  Tech  World",  a  retrospective  portrait  is 
persentcd  of  a  world  whcih  has  seemingly  become  devoid  of  humanilty  and  true  liveral  education.  "The 
society  suffers  from  a  loss  of  skills  for  living,  a  loss  of  humanity  and  a  growing  lack  ..  ;kills  for  dealing 
with  a  gloval  society,  a  chinging  world  for  managing  technology  and  deciding  whether  it  will  be  master  or 
slave"  (Nelson,  10). 

So  far,  it  is  Nelson's  contention  that  society  has  opted  for  the  role  of  slave.,  Learning  has 
witnessed  a  profound  shift  from  "classical  education"  to  a  process  of  mere  training.  As  a  result,  many  lack 
basic  reading,  cognitive,  and  spelling  skills  upon  graduation  from  high  school  (Nelson,  7). 

Vocabulary 

It  is  certainly  no  secret  that  educational  technologists  have  their  own  jargon,  much  like  any  other 
field  or  discipline.  There  has  been  concern  in  recent  years  that  the  very  terms  technology  uses,  as  well  as 
the  dialogue  created,  seems  to  isolate  people  from  one  another.  Sally  Jorgenson  once  conducted  interviews 
regarding  this  matter  with  teacher  education  faculty  at  Indiana  University,  yielding  some  poignant 
commentary.  "The  interviewee  also  asserted  that  the  machine  talk,  the  technocratic  language  of  1ST  is 
threatening,  alienating,  and  totally  unnecessary.  "  'Humans  don't  talk  like  that,'  he  said"  (Jorgenson,  268). 

James  Finn  has  also  noted,  in  "Dialogue  in  Search  of  Relevance"  that  technology's  very  dialogue 
has  helped  create  an  atmosphere  of  "impersonal  control  over  people." 

Remedies  for  this  particular  ailment  have  not  come  easy,  but  Finn  has  offered  one  practical 
suggestion.  "Unless  we  listen  to  what  some  of  our  bright  young  people  are  saying,  to  what  the  new  left  is 
trying  to  expound,  to  what  some  artists  are  expressing,  we,  as  educators,  may  fail  this  country,  and  all  the 
young  people  in  it"  (Finn,  143).  Although  Finn's  critique  was  aimed  primarily  at  the  college  educational 
system,  many  of  his  ideals  appear  relevant  to  the  K-12  classroom  as  well. 

Science,  Technology,  and  Literature 

Perhaps  one  of  the  least  addressed  aspects  of  technology  as  it  related  to  education  is  in  the  area  of 
science.  Because  technology  is  in  such  a  constant  state  of  flux,  many  of  the  new  innovations  developed 
through  the  fusing  of  these  two  related  fields  appear  increasingly  normal.  The  seventies  and  eighties 
brought  the  world  a  collection  of  art  and  literature  brimming  with  science  fiction  and  technology  related 
themes.  Authors  such  as  Ray  Bradbury  (Fahrenheit  45  1),  Michael  Crkhton  (Jurassic  Park),  and  Arthur  C. 
Clarke  (2001 :  A  Space  Odyssey)  have  carried  on  the  literary  tradition  of  Orwell  by  daring  to  envision  the 
dangers  of  the  dehumanized  society  in  their  writings. 

While  many  interpret  th.se  pop  culture  opuses  as  mere  pleasure  reading,  others  have  realized 
elements  of  reality  apparent  in  them  all.  In  1987,  Randy  Nichols  noted  that  Educational  Technology  has 
the  possibility  of  mutating  into  new  potential  categories,  such  as  Educational  Bio-Technology,  The 
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proposed  sub-field,  as  outlined  in  his  article,  sounds  a  great  deal  like  "human  engineering."  "The  key 
characteristic  is  physical  invasion  of  the  body,  though  psychological  changes  certainly  occur  also. 
Educational  bio-technologists  could  consist  of  implanting  microprocessors  in  people  or  giving  food  and 
drugs  to  people  to  change  some  aspect  of  their  education"  (Nichols,  128). 

Some  Conclusions 

The  critical  research  seems  to  indicate  that  there  are  no  easy  answers  to  the  increasing  problem  of 
dehumanization  that  confronts  us  each  day  both  in  society,  and  in  our  K-l 2  educational  system  The 
monumental  growth  of  technology  as  a  field  has  brought  many  issues  into  question  regarding  our  own 
humanity,  literacy,  emotions,  skills,  and  educational  experiences.  The  critics  seem  io  indicate  that  the 
abuse  of  technology  has  caused  a  chaotic  trend  of  dehumanization  since  the  fifties,  and  educational  standards 
continues  to  plummet. 

The  critics  essentially  have  noted  that  technology  is  not  an  end  in  itself,  and  therefore,  should  not 
be  regarded  as  such.  Objectivity  has  all  but  replaced  subjectivity  in  the  curriculum,  and  learning  has 
become  a  passi  ve  process  rather  than  an  active  one.  The  acquiring  of  knowledge  has  loo  often  been 
confused  with  rote  memorization,  drill  and  practice  tests,  and  other  devices  which  seem  to  stunt  the 
cognitive  and  reasoning  abilities  which  must  be  nurtured  in  students. 

As  a  result,  students  have  become  the  by-product  of  an  educational  hierarchy  which  establishes 
what  will  or  will  not  be  learned.  This  hierarchy  has  manifested  itself  in  various  forms,  including  high 
powered  school  administrations,  big  business,  and  most  notably,  the  military.  Though  some  have  called 
for  radical  revisionism  in  the  educational  process,  many  indications  seem  to  suggest  that  their  attempts  were 
futile.  Poor  interacMon,  discussion,  and  literacy  skills  are  just  some  of  the  problems  facing  students  as  a 
result  of  this  imbalanced  technological  structuring. 

Pear,  isolation,  submission,  and  loss  of  teacher  and  student  identity  seem  to  be  recurring  themes  contained 
in  much  of  the  critical  theory  concerning  dehumanization.  When  examining  the  work  of  the  critics  as  a 
whole,  there  seems  to  be  no  question  that  more  research  needs  to  be  done  in  this  particular  area. 
Compromise  between  liberal  education  and  technology  is  a  start,  but  not  a  direct  cure-all  for  the  many 
complexities  which  dehumanization  presents. 
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Although  many  view  computers  purely  as  technological  tools  to  be  utilized  in  the  classroom  and 
workplace,  attention  has  been  drawn  to  the  social  differences  computers  perpetuate,  including  those  of  race, 
class,  and  gender.  (Campbell,  1984;  Damarin,  1989;  Huff  et  al.,  1987)  These  differences  are  the  result  of  a 
combination  of  many  factors.  Arguments  have  ranged  from  the  view  of  computing  as  a  culture  to  the 
content  of  the  software  that  the  computers  utilize.  This  paper  focuses  on  gender  and  computing  by 
examining  recent  analyses  in  regards  to  content,  form,  and  usage  concerns. 

The  content  of  educational  software  packages  has  been  critically  examined  to  reveal  partiality  in 
their  design  (Huff  and  Cooper,  1987;  Biraimah,  1989;  DeVaney,  1993;  Rosenthal  et  al.,  1988;  Benoitet 
al.,  1991).  This  class  of  critiques  encompasses  differences  in  software  design,  concern  for  race, 
socioeconomic  class,  and  gender  stereotypes  in  the  overall  text  of  the  software,  and  the  lack  of  quality 
evaluation  of  educational  software  to  identify  gender  bias. 

Other  research  has  focused  on  the  machine  in  it's  most  basic  form,  and  have  shown  that  it  has  the 
same  effects  (Turkle,  1984:  Damarin,  1993;  Bush,  1983).  These  studies  are  rooted  in  the  theory  that 
microcomputers  have  been  developed  through  mathematics  and  science,  typically  viewed  as  masculine 
fields.  Therefore,  there  is  an  inherent  bias  toward  male-dominated  thinking.  As  a  result,  the  way  the  user 
approaches  the  computer  can  have  an  effect  on  his/her  success  or  failure. 

Finally,  the  question  of  who  is  using  computers  and  how  they  are  being  used  has  raised  the  issue 
of  access  (Campbell,  1984;  Hawkins,  1987;  Lockheed,  1985;  Nelson  et  al.,  1990).  In  order  to  ensure 
equity  in  microcomputer  use,  all  students  must  have  equal  opportunity  to  learn  about  and  use  computers. 
While  it  may  be  apparent  how  differences  in  socioeconomic  class  would  allow  for  these  discrepancies  in 
access,  issues  of  render  become  apparent  upon  examination  when  coupled  with  the  fact  that  most  computers 
were  initially  associated  with  mathematics  and  science  departments.  This  has  effected  the  way  computing  is 
taught,  the  location  of  the  computers,  the  integration  of  computers  into  schools,  and  the  uses  of  computers 
in  the  schools. 

Content 

Concerns  regarding  educational  software  content  have  been  of  interest  since  the  incorporation  of 
microcomputers  into  classrooms.  While  some  practicing  teachers  may  find  it  feasible  to  design  software  for 
their  classrooms  using  a  variety  of  programming  languages  and  authoring  systems,  the  reality  is  that  most 
teachers  have  to  use  preexisting  software  packages.  As  a  result,  there  is  concern  for  the  quality  of  this 
software,  as  well  as  the  lack  of  appropriate  means  for  evaluating  and  detecting  gender  bias.  A  sampling  of 
the  critics  of  software  design  methods  will  be  examined  first. 

Huff  and  Cooper  (1987)  conducted  a  study  regarding  software  and  it's  design.  The  purpose  was  to 
specifically  test  for  gender  differences  in  software  designed  by  experienced  educators.  This  exercise  led  to 
interesting  results.  Regardless  of  the  gender  of  the  software  designer,  there  were  differences  in  the  programs 
designed  for  male,  female,  and  a  mixed  class  of  male  and  female  students. 

The  software  designed  for  males  was  more  game-oriented  and  action-based.  The  software  which 
was  designed  for  female  students  was  more  tool-oriented,  reflecting  their  recognition  of  the  computer  as  a 
learning  tool.  While  this  is  consistent  with  other  research  of  this  nature  (Turkle,  1984),  what  is  truly 
shocking  is  that  the  software  that  was  designed  for  the  heterogeneous  group  was  most  similar  to  that 
designed  for  male  students,  i.e.  game-oriented.  What  this  indicates  is  that  there  is  a  perceived  difference 
between  the  types  of  software  that  male  and  female  students  prefer  by  teachers  designing  software. 

It  is  interesting  to  note  that  those  involved  in  Huff  and  Cooper's  experimental  design  were 
experienced  classroom  teachers.  These  are  individuals  expected  to  be  concerned  with  the  format  of 
educational  software  in  regards  to  interest  groups  and  should  therefore  be  most  conscientious  in  their  i.esign. 

The  actuality  of  educational  software  design  is  that  although  the  audience  may  be  mixed,  the 
programs  are  designed  with  male  characteristics.  There  have  been  numerous  studies  which  coincide  with 
these  findings  (Kiesler  et  al.,  1985,  Hawkins,  1987),   If  the  software  being  used  in  the  classroom  is 
designed  with  a  particular  sex  in  mind,  and  it  is  the  "wrong  sex"  for  the  user,  then  anxiety  level  could  rise 
and  learning  decline. 
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What  Huff  and  Cooper  have  suggested  as  a  possible  solution  is  that  software  be  designed  that  has 
game  and  tool  characteristics  so  that  it  will  be  appealing  to  both  sexes.  This  solution  is  a  valid  one,  but  it 
does  not  take  into  account  the  preexisting  software  which  is  being  used  in  classrooms. 

Karen  Biraimah  (1989)  addresses  the  issue  of  content  in  prepackaged  educational  software  in  her 
article  "Inequalities  in  Classroom  Computer  Software."  She  suggested  that  software,  like  texts,  transmit 
knowledge,  including  values  which  are  those  of  the  designer.  The  problem  with  this  model  is  that  the 
software  designer  is  usually  a  member  of  the  societal  majority.  Due  to  this  membership,  students  are 
exposed  to  stereotypes  and  values,  with  disregard  for  those  who  do  not  hold  those  Anglo-middle  class  male 
views.  As  a  result  of  her  acceptance  of  this  concept,  the  author  surveyed  fifteen  educational  software 
packages.  Her  findings  were  congruent  with  related  studies.  (Benoit  et  al.,  1991;  Campbell,  1984; 
DeVaney,  1993;  Rosenthal  et  al.,  1991)  The  gender  and  ethnic  under-representation  which  had  been 
previously  found  in  textbooks,  held  true  in  the  software.  Biraimah  stated  her  results  as  follows:  "software, 
like  other  instructional  materials,  is  directed  towards  a  male  audience,  with  little  sensitivity  for  the 
multicultural  aspects  of  American  classrooms."  (1989,  p.  5) 

By  comparing  bias  found  in  textbooks  to  educational  software,  the  author  was  able  to  identify 
disparities  in  educational  software  which  are  currently  in  use.  Ann  DeVaney  (1993)  went  deeper  into  the 
issue  by  examining  an  individual  piece  of  software  currently  regarded  as  one  of  the  best  educational 
programs  on  the  market. 

In  her  article  entitled  "Reading  Educational  Computer  Programs",  DeVaney  examined  the  content 
of  computer  software  by  viewing  the  program  as  a  text  that  has  been  constructed  in,  and  should  be  examined 
through,  a  social  context  (1993).  By  utilizing  reception  theory,  she  analyzed  messages  contained  in  the 
text.  DeVaney  felt  that  this  text  is  the  product  of  a  larger  discourse,  one  which  belongs  to  the  soft  ware 
author(s).  Since  the  software  entices  students  into  this  larger  discourse,  it  is  important  to  examine  it's 
contents. 

DeVaney  analyzes  the  popular  software  "Where  in  the  World  is  Carmen  San  Diego?"  and  the  social 
discourse  of  which  it  is  a  part.  Her  findings  indicate  that  this  text  is  not  neutral.  On  the  contrary,  she 
found  indications  of  a  program  which  reflects  sexist,  white,  American  values.  She  theorized  this  to  the 
existence  of  two  readers  of  the  program:  the  computer  and  the  user.  Because  the  user  is  forced  to  read  the 
program  designed  to  be  read  by  the  computer,  the  user  becomes  constructed  as  a  machine.  The  solution  to 
this  problem,  as  proposed  by  the  author,  lies  in  the  finding  of  an  alternative  to  this  type  of  construction 
within  the  limitations  of  the  hardware. 

While  this  demonstrates  the  need  to  effectively  and  critically  evaluate  software  that  is  to  be  used  in 
the  classroom,  there  is  a  lack  of  methodology  in  this  regard  among  classroom  practitioners.  This  is  not  to 
claim  that  software  is  not  being  evaluated,  but  may  not  be  evaluated  in  all  necessary  dimensions.  The  bias 
contained  in  software  is  now  more  subtle  then  in  the  beginnings  of  software  development,  yet  it  can  have 
the  same  detrimental  effects.  Tnerefore,  there  is  a  demand  for  more  effective  evaluation  of  software  in 
regards  to  gender  differences. 

This  issue  was  raised  by  Rosenthal  and  Demetrulias  (1988)  in  their  study  of  pre-service  and 
practicing  teachers  and  the  effect  of  different  evaluation  forms  on  their  identification  of  software  as 
containing  bias  or  stereotypes.    The  authors  proposed  that  the  more  attention  the  evaluation  form  paid  to 
gender  bias,  the  evaluiitor  would  respond  likewise  and  also  pay  more  attention  to  bias.  The  results  of  this 
study  were  consistent  with  their  expectations  regarding  the  different  instruments  and  the  subtlety  of  the  bias. 

On  evaluation  forms  which  contained  few,  or  even  as  little  as  one,  questions  regarding  stereotypes, 
the  evaluators  were  less  likely  to  judge  the  software  as  containing  bias.  When  the  subjects  employed  the 
Expanded  Sexism  Checklist  (ESC),  developed  by  the  authors,  they  evaluated  the  same  software,  previously 
evaluated  as  stereotype-free,  as  containing  bias.  This  study  has  brought  to  the  forefront  a  need  for  better 
software  evaluation  by  those  who  plan  to  implement  it. 

The  issue  of  software  evaluation  was  examined  again  by  Benoit  and  others  (199 1).  They  conducted 
a  survey  of  career-oriented  and  educational  software  to  evaluate  the  extent  of  gender  bias.  They  found,  by 
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identifying  stereotyped  representations  of  women,  that  the  software  they  examined  contained  gender  bias. 
They  also  found  that  older  software  was  more  likely  to  contain  gender  bias  then  software  published  more 
recently. 

As  a  means  of  overcoming  this  obstacle,  the  authors  recommend  that  software  authors  implement 
the  "publisher-established  non-sexist  writing  standards  for  textbook  authors."  (Benoit  et  al.,  1991 )  This 
approach,  although  original,  is  subject  to  some  difficulty.  It  does  not  take  into  account  the  authoring 
systems  and  programming  that  amateur  software  developers  utilize.  Hence,  there  would  be  a  gap  in  the 
software  used  in  the  classrooms,  dependent  upon  the  developer's  standard. 

Form 

Although  biased  software  can  be  viewed  as  the  primary  concern  in  educational  computing,  there  are 
other  issues.  Regardless  of  the  content  of  the  software,  students  may  be  exposed  to  stereotyping  in  the 
most  elementary  form  of  the  computer.  This  includes  the  students'  approaches  to  computing  and 
programming  languages  and  the  overall  computing  atmosphere,  which  is  generally  associated  with 
mathematics  and  science  departments. 

Sherry  Turkle  (1984)  examined  the  way  that  boys  and  girls  approach  computer  programming.  The 
views  of  males,  who  tend  to  see  programming  as  a  logical  activity,  have  dominated  what  is  taught  in 
introductory  computer  courses.  This  approach  is  generally  accepted  as  "good"  programming.  Turkle  has 
found  however,  that  the  sexes  approach  programming  very  differently  and  identified  these  differing 
approaches  as  part  of  the  difficulty  young  girls  encounter  when  learning  programming.  The  boys  tended  to 
program  by  a  highly  valued  top-down  approach,  which  requires  breaking  the  task  down  into  smaller  tasks 
and  the  prior  planning  of  each  of  these  individual  tasks.  The  girls,  on  the  other  hand,  tended  to  approach 
programming  with  what  the  author  calls  a  "soft  mastery"  approach,  which  involves  negotiating  with  the 
computer  at  the  design  level.  Because  most  young  girls  are  discouraged  from  using  this  approach,  they 
come  to  reject  programming.  But,  as  Turkle  (1984)  stated: 

"Thus,  some  girls  who  reject  programming  are  not  rejecting  the  computers  as  such  or 
programming  as  such.  They  are  rejecting  an  intellectual  style  that  has  become 
indissociable  from  it  in  the  public  mind  and  the  minds  of  teachers."  (p.  49) 

The  author  goes  on  to  draw  parallels  between  computer  science  and  other  sciences  where  hard 
mastery  is  the  preferred  and  the  "hard  masters"  define  the  field.  She  challenges  that  if  given  the  opportunity 
to  explore  the  computer  in  their  own  unique  way,  girls  may  open  the  door  to  the  study  of  other  formal 
systems,  such  as  mathematics  and  science. 

The  conclusions  that  can  be  drawn  from  this  research  is  that  girls  prefer  not  to  use  computers  when 
they  are  generally  associated  with  mathematics  and  science.  Allowing  girls  to  learn  computing  in  their  own 
style  will  give  them  the  necessary  success  to  move  forward  in  other  areas.  In  this  regard,  Sherry  Turkle 
viewed  computers  as  the  ga'.eway  for  women  into  mathematics  and  science.  Conversely,  Suzanne  Daman n 
(1989)  viewed  mathematics  as  the  gateway  into  computers.  She  resolves  that  given  the  opportunity  to 
practice  the  necessary  mathematical  skills,  girls  can  be  successful  in  computing. 

Suzanne  Damarin  (1989)  sees  the  affect  computers  have  on  women  both  in  school  and  in  the 
workplace  as  a  result  of  their  inexperience  with  computers,  which  is  largely  due  to  the  mathematics  which 
is  involved  in  their  use.  The  large  numbers  of  women  that  leave  >nathematical,  scientific,  and  computer 
rtudies.  coupled  with  the  fact  that  mathematics  and  formal  logic  are  central  to  the  computer's  functioning, 
brings  Damarin  to  urge  that  young  girls  be  given  the  opportunity  to  learn  about  and  practice  these  skills. 

Damarin  then  identifies  two  factors  which  must  be  taken  into  account:  affective  equity  and 
evaluation  equity.  The  former  requires  that  instruction  be  mindfu.  to  the  "attitudes,  feelings,  and 
preferences"  of  students.  The  latter  encourages  a  move  away  from  placing  value  upon  a  narrow  set  of 
answers  toward  "the  diversity  of  ways  in  which  computers,  math,  and  science  can  be  understood,  used,  and 
employed"  (Damarin,  1989).  In  the  computer  classroom,  this  means  placing  value  on  the  good  qualities  of 
a  program,  instead  of  placing  value  on  "good''  programming. 


One  of  the  suggestions  for  the  overcoming  of  this  obstacle  is  to  introduce  girls  into  the  world  of 
computing  as  a  whole.  This  is  based  on  the  hypothesis  that  if  girls  are  comfortable  in  the  culture  of 
computing,  they  will  be  more  successful.  Corlann  Gee  Bush  (1983)  suggested  that  analyzing  the  very 
personal  context  of  technology  use,  including  the  use  and  discussion  of  computing  in  the  user's  terms, 
would  allow  for  a  rethinking  of  the  user  context,  and  would  result  in  an  "understanding  of  the  effects  of 
technological  change  on  women's  lives."  (p.160). 

The  context  of  computer  usage  has  become  more  personal  over  time  as  the  trend  in  many  schools 
today  is  the  introduction  of  the  computer  into  many  content  areas,  and  even  as  a  stand-alone  subject.  This 
provides  many  opportunities  for  students  to  find  and  become  comfortable  with  computing.  As  a  result, 
there  is  a  shift  away  from  the  direct  association  of  computers  with  mathematics  and  science,  yet  the  gender 
differences  persist. 

In  1985,  Kiesler  and  Sproul  further  examined  this  conflict.  The  authors  found  that  while  there  was 
increasing  availability  of  computers  in  society,  there  was  not  equal  usage  by  all.  The  authors  attributed  this 
disparity  to  the  perception  of  computing  as  a  culture.  It  is  because  of  the  existence  of  this  culture  that 
computer  training  should  include  lessons  on  living  within  this  culture,  as  well  as  skills  and  knowledge.  As 
it  existed  at  the  time  of  the  writing,  computing  was  male  dominated  and  typically  transmitted  through 
males  both  in  and  out  of  schools.  The  authors  stated  that  as  long  as  girls  are  unfamiliar  with  this  culture, 
they  will  remain  unfamiliar  with  the  hardware.  Furthermore,  since  they  do  not  believe  that  there  is 
anything  intrinsic  that  would  make  computers  more  difficult  for  girls,  they  see  a  need  for  software  which 
allows  all  children  to  have  computer  ability. 

Usage 

Third,  there  is  a  cause  for  concern  in  usage,  both  qualitatively  and  quantitatively.  While  some 
students  may  have  difficulty  getting  close  to  a  computer  physically,  others  may  find  it  difficult  getting 
close  intellectually.  This  barrier  can  be  created  by  a  variety  of  situations,  among  them  are  the  way  in  which 
computers  are  introduced  and  how  they  are  used  in  the  learning  environment. 

Patricia  Campbell  (1984)  examined  usage/access  concerns  and  she  found  that  while  approximately 
half  of  all  students  are  female,  two-thirds  of  the  students  learning  computers  are  male.  These  figures 
generally  hold  true  in  and  outside  the  classroom.  Campbell  suggested  that  the  way  that  computers  are 
introduced  is  a  contributing  factor  to  these  differences. 

First,  computers  were  often  viewed  as  "math  machines"  and  therefore  any  math  anxiety  students 
may  have  had  was  transferred  to  the  computer.  Also,  teachers  typically  used  boys  to  help  introduce  the 
computer  to  the  class.  This  was  due  to  the  perception  that  boys  have  prior  experience  with  computers,  or 
are  viewed  as  more  mechanical.  Lastly,  where  there  are  limited  computers  for  a  large  group  of  students,  the 
more  aggressive  students,  typically  the  males,  used  the  computers. 

As  a  result  of  the  combination  of  these  factors,  young  girls  were  using  computers  less  than  their 
male  counterparts.  As  more  people,  especially  teachers,  became  aware  of  these  differences,  girls  gained 
physical  accessibility  to  the  hardware  and  software.  The  focus  then  became  an  examination  of  deeper  issues. 

Jan  Hawkins  (1987)  examined  the  reports  from  various  research  endeavors  regarding  gender  and 
computers.  She  found  that  both  the  organization  of  the  learning  and  the  role  the  computer  played  in  the 
learning  environment  influenced  computer  use  by  males  and  females.  Due  to  the  link  between  computers 
and  mathematics  and  science,  the  computers  were  often  integrated  into  one  of  these  areas.  The  result  was 
that  girls  encounter  difficulty  utilizing  the  computers  because  of  the  view  of  these  areas  as  male  domains. 

Other  research  Hawkins  examined,  studied  the  role  of  nature,  nurture,  and  society  at  large  in 
influencing  girls  use  of  computers.  The  general  conclusions  were  that  the  characteristics  of  a  task  heavily 
influence  the  amount  girls  used  the  computers.  Again,  girls  were  less  likely  to  use  computers  when 
programming  was  being  taught,  but  were  more  likely  to  use  them  when  word  processing  was  the  central 
task.  In  regards  t»  mathematics  and  science  software,  girls  preferred  those  that  allowed  students  to  work 
cooperatively  and  the  goals  of  which  were  not  blatantly  scientific. 
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These  articles  were  written  early  in  the  computer-education  relationship  and  the  authors  urged  the 
readers  to  be  aware  of  the  formation  of  stereotypes  because  it  was  early  enough  to  curb  them.  As  access  and 
usage  increased  there  was  a  shift  away  from  concerns  regarding  only  the  amount  of  time  which  was  spent  on 
computers.  One  study  which  was  under  way  at  the  time  found  that  even  as  students  used  computers  more 
and  had  increased  exposure  to  them,  the  differences  in  attitudes  between  the  sexes  did  not  significantly 
change.  (Krendl  et  al.,  1989)  On  the  contrary,  confidence  and  interest  in  computers  by  girls  were  unaffected 
by  time.  It  became  an  issue  of  benefit  equity,  people  began  to  examine  why  gender  differences  persisted 
even  as  the  amount  of  time  using  computers  leveled  out. 

Lockheed  (1985)  examined  why  women  and  girls  were  not  using  computers  and  attempted  to 
explain  why.  She  resolved  that  the  disparity  was  due  to  the  differences  in  the  ways  in  which  boys  and  girls 
use  computers.  And  the  manner  in  which  they  are  using  them  has  led  to  the  defining  of  computing  as  a 
male  domain. 

As  was  stated  in  much  of  the  research  on  the  issue  of  usage,  boys  are  more  likely  then  females  to 
use  the  computer  for  game-playing  and  programming.  (Lockheed,  1985)  Conversely,  when  the  computer 
was  used  for  other  applications,  there  were  little  or  no  differences  in  use  by  gender.  Unfortunately,  much  of 
what  is  viewed  as  valuable  computer  use  is  programming,  resulting  in  the  de-emphasizing  of  girls  use  of 
computers. 

In  the  article  "The  Computer  Gender  Gap:  Children's  Attitudes,  Performance,  and 
Socialization"  the  authors  examine  gender  differences  in  terms  of  access  and  performance  outcomes.  (Nelson 
&  Watson,  1 990-91 )  While  there  has  historically  been  gender  differences  in  education,  the  question  remains 
of  whether  the  new  technologies,  i.e.  computers,  will  diminish  or  perpetuate  existing  gaps.  Since  the 
introduction  of  computing  into  all  areas  of  school,  the  authors  believe  the  transference  of  math  anxiety  to 
the  computer  has  decreased.  What  are  now  viewed  as  the  differentiating  factors  are  the  influence  of  the 
family,  gender  bias  in  the  software,  and  the  influence  of  the  teacher. 

Conclusions 

From  the  research  examined,  three  key  inferences  may  be  drawn  in  regards  to  interaction  with 
computers,  research  methods,  and  research  questions.  The  writings,  dating  from  1983  -  1993,  examined  in 
this  analysis  contain  a  recurring  call  for  varying  types  of  educational  computing  experiences.  These  would 
allow  for  differing  ability  and  interest  levels,  as  well  as  varying  types  of  interactions.  (DeVaney,  1993; 
Ellsworth,  1988;  Rosenthal  et  al.,  1988;  Campbell,  1984;  Nelson  et  al.,  1990-1) 

Also  threaded  throughout  these  readings  is  a  common  research  methodology.  Most  of  the 
examinations  of  software  content  and  computer  usage  were  studied  through  the  standard  research  methods 
such  as:  controlled  studies  (Rosenthal  et  al.,  1988;  Huff  et  al.,  1987),  surveys  (Demetrulias,  1985;  Krendl 
et  al.,  1989;  Lockheed,  1985),  and  observations  (Kiesler  et  al.,  1985). 

Also  evident  in  these  readings  are  research  questions  which  vary  in  their  approach.  As  the 
discrepancy  between  gender  and  computing  came  to  the  surface,  the  initial  issue  was  how  the  hardware  wax 
being  used  in  the  classrooms  (Demetrulias,  1985;  Hawkins,  1987;  Mclnerney  et  al.,  1986).  This  approach 
addressed  issues  of  accessibility  to  computers  and  equal  usage  time  by  both  sexes.  As  this  concern  was 
identified  and  then  resolved,  it  became  apparent  that  there  were  other  concerns  to  be  addressed. 

Researchers  (Biraimah,  1989;  Rosenthal  et  al.,  1988;  Benoitetal.,  1991)  began  focusing  on  the 
software  which  was  being  currently  utilized  in  classrooms.  Their  findings,  discussed  earlier,  established  a 
concern  for  the  content  of  this  software.  Many  software  packages  were  found  to  have  varying  degrees  of 
stereotyping  and  bias  within  them. 

Finally,  others  studied  the  context  in  which  the  hardware  and  software  were  being  used  (Krendl, 
1989;  Ellsworth,  1988).  These  studies  found  that  the  environment  in  which  the  computer  was  used, 
regardless  of  equal  access  and  representation  in  software,  effected  the  performance  of  students  in  regards  to 
their  gender. 
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The  culmination  of  these  various  writings  has  come  to  the  current  emphasis  for  an  examination  of 
educational  computing  from  a  social/societal  perspective  rather  than  a  technological  one  (Huff  et  al. ,  1987; 
Kiesler  et  al.,  1985).  As  has  been  the  history  of  the  gender-educational  computing  field  to  be  reactive  to 
such  concerns,  addressing  this  relationship  through  new  methods  may  help  to  identify  and  resolve,  or 
perhaps  prevent  further  areas  of  conflict.  The  continuing  growth  of  the  Field  allows  for  such  a  change  in 
perspective  in  further  studies. 
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As  machines  have  become  a  more  common  part  of  daily  life  through  the  passage  of  time,  the  idea  that  the 
line  separating  man  and  machine  is  slowly  fading  has  become  more  popular  as  well.  I  would  like  to  discuss 
three  critics  of  this  change  through  their  most  famous  works. 


Mary  Shelley's  impact  on  this  school  of  thought  is  obvious  to  this  day.  Although  her  most  famous  work, 
Frankenstein,  has  been  interpreted  in  numerous  ways,  analyzed  as  statement  about  everything  from  sexual 
tension  to  a  woman's  role  in  society,  one  of  the  most  popular  views  on  the  book  is  as  a  criticism  of 
science's  impact  on  people.  The  plot  of  the  book  revolves  around  a  man  of  science  and  how  his  dreams  of 
bettering  man  through  technology  go  terribly  wrong.  This  in  itself  could  be  interpreted  as  a  statement 
about  uncontrolled  progress,  but  when  the  characters  in  the  book  are  looked  into  more  deeply  a  deeper 
criticism  becomes  evident.  She  describes  Dr.  Frankenstein  as  a  man  devoid  of  spiritual  foundation,  lacking 
in  any  belief  in  the  world  outside  the  tangible.  Dr.  Frankenstein  describes  his  beliefs  during  his  narrative  in 
chapter  4,  "...my  father  had  taken  the  greatest  precautions  that  my  mind  should  be  impressed  with  no 
supernatural  horrors  ..."  and  later  saying, "Darkness  had  no  effect  upon  my  fancy;  and  a  church  yard  was  to 
me  merely  the  receptacle  of  bodies  deprived  of  life...."  The  void  left  by  his  lack  of  spiritualism  is  filled 
with  the  drive  for  scientific  inquiry,  and  thus  it  is  the  science  that  has  become  such  a  big  part  of  his  psyche 
that  eventually  leads  him  into  peril. 

In  Frankenstein  Shelley  laid  the  groundwork  for  the  concerns  of  many  in  the  future,  and  created  one  of  the 
most  common  themes  in  science  fiction  today.  As  Terrence  Holt  wrote  in  "Frankenstein  as  Science 
Fiction"  describing  the  most  bitter  irony  of  the  novel  "The  inventor's  hope  of  accelerating  evolution  and  the 
ironic  devolution  of  both  creature  and  creator  not  only  reflect  the  central  strain  of  nineteenth  century 
biological  theory,  but  also  introduce  two  enduring  themes  of  modern  science  fiction,  evolution  and  its 
discontents."  What  she  did  is  create  a  work  that  many  people  clung  to  at  that  time,  for  the  middle  class  that 
was  currently  undergoing  industrialization  was  looking  for  a  work  like  Shelley's  to  show  them  that  their 
fears  were  justified,  "In  short,  Frankenstein  sings  the  litany  of  a  middle  class  undergoing 
industrialization,...".  Shelley  accomplished  a  remarkable  feat  with  this  work,  making  it  accessible  for  the 
masses  during  her  time  period  while  being  truly  ahead  of  her  time.  The  idea  that  man  and  science,  or  man 
and  machines  combination  would  be  the  next  step  in  evolution,  that  pervades  the  book,  is  still  a  very 
intriguing  concept,  and  she  laid  an  excellent  groundwork  for  thought  to  this  day. 

Another  work  that  came  much  later,  but  in  many  ways  bears  a  strong  resemblance  in  message,  if  not  theme 
to  "Frankenstein"  is  Marshall  McLuhan's  The  Media  is  the  Massage.  Although  Shelley  was  likely  more 
concerned  with  technology  and  industrialization  as  a  whole,  and  McLuhan  is  more  concerned  with  media, 
both  make  a  strong  statement  about  technology's  encroachment  around  and  into  man's  body  and  psyche. ' 

In  this  book  McLuhan  states:  "the  wheel  is  an  extension  of  the  foot. ..the  book  is  an  extension  of  the 
eye... electric  circuitry  an  extension  of  the  central  nervous  system",  he  attempts  to  roll  all  of  the  things  that 
are  common  in  a  persons  life  into  that  person's  being,  to  him  there  seems  to  be  little  use  for  distinctions 
between  man's  physical  makeup  and  man's  complete  make  up  at  all.  This  is  an  interesting  and  relatively 
radical  idea,  but  it  was  his  predictions  made  from  this  that  really  drew  attention  to  the  work.  McLuhan 
explains  "Media,  by  altering  the  environment,  evoke  in  us  unique  ratios  of  sense  perceptions.  The 
extension  of  any  one  sense  alters  the  way  we  think  and  act-  the  way  we  perceive  the  world."  An  example 
would  be  crediting  the  invention  of  type  for  the  growth  of  linear  thinking,  which  in  turn  would  mean  if 
writing  had  evolved  another  way  (like  hieroglyphics  for  example)  a  different  thought  process  most  likely 
would  have  developed. 

It  is  from  here  that  he  makes  the  prediction  about  man's  future  that  drew  the  most  negative  reaction.  When 
he  wrote:  "At  the  high  speeds  of  electric  communication  a  purely  visual  means  of  apprehending  the  world 
are  no  longer  possible;  they  are  just  too  slow  to  be  relevant  or  effective",  in  other  words,  technology  must 
improve  or  replace  vision  in  order  for  man  to  continue  growth.  Many  interpreted  this  as  meaning  that 
books  and  literature  as  known  today  and  as  they  have  been  known  in  the  past  are  no  longer  necessary  and 
will  fade  into  obsolescence.  This  statement  acquired  him  scores  of  critics  from  all  areas  of  English  and 
communications.  These  individuals  felt  as  though  McLuhan  was  attempting  to  replace  the  soul  of 
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communication  with  technology  in  much  the  same  way  that  science  replaced  the  soul  of  Dr.  Frankenstein 
in  Shelley's  work. 

Samuel  Florman's  The  Existential  Pleasures  of  Engineering  is  related  to  these  first  two  works,  as  he  is 
concerned  with  the  relationship  between  man  and  machine.  He  differs  from  my  first  two  subjects 
significantly,  as  they  spoke  in  speculation,  Florman  speaks  about  what  has  happened.  It  is  r:~-  *.o  see  how 
engineers  as  whole,  being  closest  to  technology  would  be  the  most  effected  by  it,  and  thus  "1   ■  Existential 
Pleasures  of  Engineering  finds  its  niche.  Throughout  the  book  Florman  makes  no  real  distinction  between 
Engineers  and  their  work,  and  even  make  efforts  to  blur  the  line  between  the  two. 

Instead  of  speaking  about  what  could  happen  if  man  and  machine  become  too  close,  he  speaks  about  what 
has  happened  due  to  this  combination.  A  nostalgic  tone  pervades  the  beginning  of  the  book,  speaking 
about  the  glory  days  when  the  creations  of  the  engineer  brought  him  acclimation  and  pride,  in  contrast  to 
the  blame  and  mistrust  of  today.  It  is  interesting  to  see  how  he  admonishes  those  who  criticize  the 
engineers,  at  one  point  trying  absolve  the  engineer  of  responsibility  from  the  results  of  their  work,  yet  later 
on  trying  to  strengthen  the  connection.  An  example  is  his  refute  of  criticism  from  antitechnologists,  where 
Florman  disregards  their  attitude  because  they  speak  of  technology  "as  if  it  had  and  existence  of  its  own." 

I  regard  The  Existential  Pleasures  of  Engineering  as  a  book  for  the  new  man  shaped  and  built  with 
machines,  a  book  that  begins  with  the  premise  that  the  engineer  and  his  work  are  one,  and  then  goes  from 
there.  Deliberately  or  not,  it  gives  a  remarkable  account  of  how  the  things  predicted  in  the  past  have  come 
true.  An  excellent  example  is  the  relationship  between  technology  and  the  morals  of  the  engineer. 
Florman  says:  "I  submit  that  study  of  the  liberal  arts  will  rob  him  of  his  innocence,  stain  his  character, 
make  him  less  'moral'-  or  at  least  less  naive."  If  this  is  true,  it  is  an  incredible  indication  of  how  the 
morals  of  the  engineer  are  formed,  apparently  from  the  things  that  they  build,  instead  of  from  the  society 
they  build  for. 

As  the  first  generation  raised  with  computers  from  birth  reaches  maturity  I  think  that  the  precedents  set  in 
the  past  will  prove  very  influential  in  the  future.  Whether  or  not  man  will  lead  machine  or  machine  will 
lead  man  has  yet  to  be  decided.  It  will  be  very  interesting  to  see  to  what  extent  the  predictions  and  ideas 
given  by  these  three  authors  develop  in  the  future. 
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Abstract 


This  paper  presents  the  theoretical  framework  for  a  new  course  for  teacher  education.  It 
describes  how  computers  have  been  used  in  education  and  identifies  factors  that  contribute  to  the 
unrealized  potential  of  technology  use  in  schools.  This  new  course  is  consistent  with  a  constructivist 
view  of  learning,  new  educational  goals,  and  a  view  of  the  role  of  technology  as  a  tool  that  can  support 
the  attainment  of  educational  goals  and  increased  student  learning. 

Throughout  the  twentieth  century,  each  time  a  new  form  of  media  or  technology  appears  on  the 
market,  predictions  have  been  made  concerning  the  vast  changes  that  it  will  undoubtedly  bring  to 
classrooms  and  to  the  educational  process.  The  arrival  of  personal  computers  was  no  exception. 
Researchers  and  futurists  predicted  great  changes  to  come  due  to  the  presence  of  computers  in  our  schools. 
The  envisioned  changes  were  not  realized,  however,  through  the  mere  existence  of  machines  in  classrooms 
or  school  buildings.  In  fact,  change  has  been  so  slow  that  in  many  schools  computers  have  yet  to  move 
from  administrative  offices  to  classrooms. 

Computers  are  having  an  enormous  and  pervasive  impact  on  our  business  world  and  on 
society.    The  scope  of  their  effect  has  focused  more  attention  on  classroom  teachers'  use  of 
technology  and  the  perceived  failure  of  education  to  fully  utilize  computers  in  the  instructional  process. 

A  Brief  History  of  Computer  Use  in  Education 

Many  reasons  are  cited  for  why  the  adoption  and  use  of  computers  in  schools  and  classrooms 
has  not  matched  predictions.  Primary  among  these  are  expense  of  hardware,  as  well  as  lack  of  adequate 
software  and  teacher  training.  These  are  definitely  real  deterrents,  but  in  some  schools  that  have  the 
hardware,  software,  and  training,  computers  are  still  sitting  in  closets  or  in  the  back  of  classrooms, 
unused. 

Teaching  the  technology  is  the  obvious  change  that  accompanies  the  introduction  of  any  new 
technology.  With  the  introduction  of  computers,  some  schools  that  identified  the  importance  of  computer 
skills  chose  to  focus  on  teaching  computers  as  a  separate  curriculum  while  others  carefully  planned  and 
integrated  computer  literacy  objectives  into  the  curriculum.  Still  others  chose  to  implement  some 
combination  of  these  approaches  (OTA,  1988). 

When  personal  computers  first  appeared  in  classrooms,  their  instructional  use  took  one  of  two 
roles.  The  first  use  was  to  teach  about  the  technology  itself,  including  computer  literacy  and  programming 
language  courses.  The  second  use  was  to  support  the  current  curriculum  through  the  use  of  educational 
software  such  as  tutorials  and  drill  and  practice.  In  most  cases  computers  were  used  to  support  traditional 
goals,  using  established  methods  with  as  little  change  as  possible  to  the  classroom  setting  or  well- 
established  routines  (Olson,  1981,  1980).  In  such  settings,  the  computer  did  little  to  change  the  curriculum 
or  to  truly  impact  education. 

With  the  growing  expectation  for  the  instructional  use  of  computers  in  classrooms  and  the 
identified  need  for  teacher  training,  colleges  and  universities  began  to  offer  computer  literacy  courses  for 
teachers.  The  evolution  of  these  courses  has  paralleled  that  of  computer  literacy  courses  in  schools. 
Computer  literacy  courses  have  evolved  from  courses  that  taught  about  computers  (their  history  and 
component  parts)  and  introductory  computer  programming,  to  courses  that  have  replaced  programming 
languages  with  basic  software  applications  (word-processing,  spreadsheets  and  databases).  These  courses 
have  attempted  to  address  how  these  skills  can  be  used  to  support  instruction  and  how  the  use  of  computers 
can  be  integrated  within  the  curriculum.  They  have,  however,  supported  teachers'  views  of  computers  as 
additional  content  that  needs  to  be  taught  within  an  already  crowded  curriculum. 
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Factors  Contributing  to  the  Use  of  Technology  in  the  Classroom 


The  implementation  of  technology  as  an  instructional  tool  is  a  form  of  educational  change  and 
needs  to  be     wed  and  treated  consistently  with  what  we  have  learned  concerning  change.  The  use  of 
technology  is  tied  to  the  reform  of  education  (AIT,  1994)  in  additional  ways  as  well.  It  can  support  the 
attainment  of  the  educational  goals  that  have  been  identified  for  success  in  the  information  age  society  of 
the  twenty-first  century.  One  lesson  that  we  must  not  forget  is  that  reform  efforts  must  deal  with  the 
system  as  a  whole. 

Teacher  development  has  lbcused  on  training  in  the  use  of  technology.  Learning  to  use 
technology  is  not  sufficient  for  teachers  to  make  this  change  in  their  teaching  styles.  Even  more  important 
than  "how-to"  training  is  educating,  teaching  "why"  (Jost  and  Schneberger;  1994).  Teachers,  like  all 
learners,  need  to  see  the  relevance  for  using  the  technology.  Teachers  need  to  understand  how  technology 
can  support  instruction  and  learning,  how  technology  can  help  us  to  do  things  (accomplish  important 
educational  goals)  that  either  would  be  very  difficult  or  impossible  to  do  without  the  technology.  This 
requires  a  change  in  seeing  the  role  of  computers  (or  technology)  as  instructional  and  learning  tools,  not  as 
add-ons  or  content.  Demonstrating  and  discussing  the  potential  of  various  technologies,  introducing  new 
uses  of  technology,  and  supporting  teachers  in  exploring  different  technologies  and  software  can  support 
teachers  in  constructing  a  different  understanding  of  their  own  discipline.  Teachers  can  benefit  through 
activities  that  support  their  reflection  of  what  they  teach  and  why. 

In  order  to  have  teachers  use  technology  in  constructive  ways  that  support  educational  change  and 
educational  goals,  we  must  first  assess  their  views  on  learning  and  instruction.  The  use  of  technology  as 
instructional  and/or  learning  tools  is  compatible  with  a  constructivist  view  of  learning  and  instruction  (Jost, 
1992).  In  research  concerning  teachers'  integrating  technology  into  instruction,  it  was  also  found  that 
teachers'  views  concerning  instruction,  learning,  and  teacher-student  roles  were  not  only  reflected  in  the 
goals  that  teachers  hold  for  their  students  but  also  affords  the  teaching  styles  they  use  and  the  role  that 
technology  can  assume  in  their  classrooms  (Jost,  1992).  Teachers  need  the  opportunity  to  reflect  on  how 
and  what  they  were  taught  compared  to  what  is  important  for  today's  students  to  learn.  They  need  to  see 
the  consistencies  between  new  goals  for  education,  new  theories  of  learning  and  instruction,  and  innovative 
learning  activities  and  methods  of  assessment. 

Teachers  must  also  construct  an  understanding  of  the  changing  roles  of  technology,  teachers, 
and  students.  Learning  environments  require  a  change  in  the  roles  of  both  teachers  and  students.  The 
teacher's  role  in  student-centered  learning  environments  is  to  guide,  stimulate,  facilitate  and  support 
students'  learning  activities.  Teachers  also  facilitate  student-to-student  interaction,  modeling  desirable 
thinking  and  learning  behaviors.  Using  technology  as  learning  tools  in  the  classroom  also  requires 
teachers  to  see  their  class,  including  themselves,  as  a  community  of  learners  (AIT,  1994). 

Support  ii  a  real  problem  that  exists  in  the  culture  of  educational  organizations.  The 
existence  of  the  basic  tools  is  not  enough.  Teachers  are  not  given  the  time  or  the  freedom  to  explore 
the  instructional  uses  of  technology  or  to  redesign  curriculum  or  lesson  plans.  Teachers  need  time  to 
experiment  and  become  comfortable  with  new  instructional  techniques  and  with  technology. 


Why  Do  We  Want  Teachers  to  Use  Technology? 

Many" educators,  researchers,  and  businessmen  have  emphasized  the  need  to  rethink  our  goals 
for  education.  Concurrent  with  the  redefinition  of  educational  goals  is  the  emergence  of  new 
assumptions  about  learning  and  instruction.  Schools  are  responding  by  creating  information-rich 
environments  for  instruction  and  administration  utilizing  technology  as  the  vehicle  for  restructuring 
education  to  meet  the  needs  and  challenges  of  our  information  society. 

The  use  of  technology  can  support  the  kinds  of  active  environments  that  research  on  learning 
suggest  are  supportive  for  enabling  students  to  construct  meaningful  knowledge.  The  role  of 
technology  within  learning  environments  is  that  of  a  learning  tool  -  providing  access  to  information, 
facilitating  inquiry  and  communication,  supporting  collaboration  and  knowledge  construction.  In 
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addition  it  can  function  as  a  productivity  tool  to  support  other  instructional  and  administrative 
activities. 

These  rich  environments  promote  active  knowledge  construction  in  authentic  and  meaningful 
contexts.  They  also  encourage  students  to  assume  a  more  responsible  role  in  their  own  learning 
(Grabinger  &  Dunlap,  1994);  support  the  development  of  collaborative  decision-making  and  problem- 
solving;  and  foster  the  development  of  research  and  meta-cognitive  skills.  Social  interaction  is  an 
important  component  in  these  environments,  supporting  cognitive  development  (Vygotsky,  1987; 
Wertsch,  1985).  This  approach  stresses  the  process  of  learning,  including  the  important  component  of 
reflection,  rather  than  the  learning  of  content  alone.  This  philosophy  is  consistent  with  preparing  a 
student  for  the  future. 

Based  on  the  assumption  that  we  believe  it  is  important  to  teach  higher-order  learning, 
thinking  and  inquiry  skills  that  utilize  technology  as  learning  tools,  we  must  prepare  today's  teachers  to 
be  capable  of  implementing  an  instructional  model  which  affords  students  the  generative  learning 
opportunities  which  support  the  attainment  of  these  educational  goals  (e.g.,  Cognition  &  Technology 
Group  at  Vanderbuilt  fCTGV],  1992).    The  research  of  both  the  CTGV  (1992)  and  Jost  (1992) 
indicate  that  for  teachers  to  implement  an  instructional  model  which  effectively  uses  technology  as  an 
instructional  tool  they  must  have  student-centeTed,  constructive  views  on  learning,  and  view  their  role 
as  facilitator  rather  than  as  disseminator.  Teacher  education,  therefore,  must  include  teaching  and 
modeling  of  a  generative  instructional  model,  the  integration  of  technology  as  a  tool,  and  a  constructive 
view  of  the  learning  process. 

Teaching  a  philosophy  or  theory  alone  is  not  enough.  Teachers  must  be  shown,  through 
modeling,  examples  of  instructional  strategies  and  learning  activities  that  put  these  ideas  into  action. 
Changing  one's  teaching  style  is  not  an  easy  task.  People  revert  to  what's  comfortable,  which  is 
teacher  control,  a  teacher-directed  (content-centered)  style.  It  is  easier  and  is  what  we  have  had  the  most 
experience  with.  For  many  teachers,  being  able  to  effectively  make  the  change  will  first  require  a 
conceptual  change  concerning  their  beliefs  about  learning  and  instruction.  A  generative  learning 
environment  promotes  a  climate  of  collaborative  inquiry  in  which  questioning  and  reflection  are 
integral  components.  Questioning  and  reflection  are  also  important  for  promoting  conceptual  change; 
therefore,  participating  in  this  type  of  learning  environment  should  support  conceptual  change. 

Teaching  with  Technology  is  a  new  course  for  teacher  education  which  links  the  use  of 
technology  with  appropriate  learning  theory  and  pedagogy.  It  focuses  on  how  students  learn  and  how 
technology  can  be  used  as  part  of  a  learning  environment  that  promotes  and  enhances  that  learning. 
This  course  is  also  part  of  a  research  project  that  can  be  characterized  as  simultaneously  working  toward 
change  and  observing  and  reflecting  on  that  process.  It  is  a  form  of  Action  Research  where  we  are  not 
only  studying  beliefs,  practices,  and  behaviors  (in  the  form  of  teacher  constructs)  but  are  also 
attempting  to  change  them  in  certain  ways. 

The  main  objective  of  this  project  is  to  create  a  learning  environment  for  teachers  and  teacher 
educators  which  will  graduate  professionals  capable  of  teaching  in  collaborative  learning  environments 
which  utilize  technology  as  instructional  and  learning  tools.  The  theoretical  framework  underlying  this 
environment  includes  assumptions  about  educational  goals  and  the  nature  of  learning.  This  framework 
is  consistent  with  the  goals  expressed  by  Resnick  &  Klopfer  (1989),  the  National  Council  of  Teachers 
of  Mathematics  (1989),  and  the  American  Association  for  the  Advancement  of  Science  (1989). 

This  R&D  project  includes  the  design  and  development  of  learning  activities  and  tools 
which  model  instructional  strategies  and  provide  inquiry  into  emerging  educational  goals,  methods  of 
assessment,  learning  activities,  and  the  role  of  technology  in  the  learning  environment.  It  also 
includes  activities  to  promote  students'  reflection  on  their  beliefs  concerning  learning,  instruction,  and 
the  role  of  technology  within  education. 
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Significance 


The  design  of  innovative  educational  environments  has  both  theoretical  and  practical 
outcomes.  The  three  main  aspects  to  this  type  of  research  include:  theory-building,  research  on 
learning,  and  design  (engineering). 

This  study  can  inform  instructional  designers  concerning  the  design  of  learning  environments 
and  the  development  of  instructional  models  for  teaching-with-technology.  It  can  also  inform  teacher 
educators  concerning  effective  methods  and  desired  outcomes  for  teacher  education  programs. 
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Abstract 

This  study  examined  how  content  considerations  influenced  the  instructional  strategy  decisions  reported  by 
novice  and  experienced  teachers.  A  total  of  152  preservice  and  153  in-service  teachers  completed  an  instrument 
which  asked  the  teachers  to  indicate  the  likelihood  they  would  include  selected  instructional  functions  in  lessons 
dealing  with  well-structured  and  ill-structured  content.  The  results  indicated  that  both  novice  and  experienced 
teachers  would  vary  their  instructional  strategies  based  on  content  structure.  Surprisingly,  their  reported 
variations  were  similar  and  consistent  with  the  literature  regarding  instruction  for  well-structured  and  ill- 
structured  content.  The  results  raise  questions  about  the  tacit  knowledge  preservice  teachers  have  about 
instruction.  The  results  also  have  implications  for  teacher  evaluation. 


Teacher  Decision  Making  Regarding  Content  Structure: 
A  Study  of  Novice  and  Experienced  Teachers 

Teachers  are  often  characterized  as  decision  makers.  For  example,  they  decide  which  topics  to  teach,  how 
the  topics  should  be  sequenced,  and  which  examples  to  use.  These  decisions  influence  instruction  and  ultimately 
impact  learning  outcomes;  they  are  based  on  information  about  learners,  subject  matter,  and  other  pertinent, 
factors  (Shavelson  &  Stern,  1981).  Shulman  &  Grossman  (1988)  extended  the  notion  of  teacher  as  decision 
maker  by  providing  a  theoretical  model  of  the  domains  of  teachers'  professional  knowledge,  including  subject 
mat'icr  knowledge  and  pedagogical  content  knowledge  (i.e.,  understanding  how  to  represent  specific  subject 
matter  topics  and  issues).  Teachers  draw  from  these  domains  as  they  plan  and  implement  instruction. 

One  particularly  important  decision  related  to  subject  matter  knowledge  and  pedagogical  content  knowledge 
centers  on  the  extent  to  which  the  to-be-learned  content  is  "structured."  Rosenshine  and  Stevens  (1986)  placed 
content  on  a  continuum  ranging  from  well-structured  to  ill-structured.  Examples  of  well-structured  content 
include  English  grammar  and  mathematics  facts;  ill-structured  content  includes  writing  poetry  and  analyzing 
social  issues.  Rosenshine  and  Stevens  indicated  that  instructional  strategies  should  be  adjusted  based  on  the 
extent  to  which  the  content  is  well-  (or  ill-)  structured. 

The  purpose  of  this  study  was  to  examine  how  content  considerations  influence  the  instructional  strategy 
decisions  reported  by  novice  and  experienced  teachers.  Specifically,  the  study  investigated  how  teachers' 
instructional  strategy  decisions  reflect  the  distinction  between  well-  and  ill-structured  content,  and  how  teaching 
experience  is  related  to  these  decisions.  It  was  hypothesized  that  experienced  teachers  would  be  more  likely  than 
novice  teachers  to  make  appropriate  instructional  strategy,  decisions  for  a  given  content  structure. 

Literature  Review 

The  literature  on  teacher  decision  making  is  broad  and  well  developed,  especially  at  the  intersection  of 
content  structure  and  instructional  planning.  The  review  which  follows  first  examines  the  role  of  teacher 
knowledge  in  instructional  planning.  Then  the  review  provides  a  more  detailed  look  at  the  literature  related  to 
content  structure  and  instructional  planning.  Finally,  attention  is  given  to  the  issue  of  how  novice  and  expert 
teachers  approach  instructional  planning. 
Teacher  Knowledge  and  Instructional  Planning 

As  mentioned  earlier,  teacher  decisions  are  based  on  the  knowledge  the  teacher  uses  in  the  process  of 
planning,  conducting,  and  evaluating  instruction. 

Thompson  (1992)  reviewed  the  literature  and  noted  that  studies  of  the  relationship  between  teachers'  beliefs 
about  teaching  and  instructional  practice  have  yielded  inconsistent  results.  In  some  cases,  (e.g.,  Shirk,  1973; 
Grant,  1984)  a  high  degree  of  agreement  was  found  between  teachers'  professed  views  of  mathematics  teaching 
and  their  instructional  practice.  In  other  studies  (e.g.,  Brown,  1985;  Cooney,  1985),  discrepancies  between 
beliefs  and  practices  were  found.  Thompson  concluded  that  "...any  serious  attempt  to  characterize  a  teacher's 
conception  of  the  discipline  he  or  she  teaches  should  not  be  limited  to  an  analysis  of  the  teacher's  professed 
views.  It  should  also  include  an  examination  of  the  instructional  setting,  the  practices  characteristic  of  that 
teacher,  and  the  relationship  between  the  teacher's  professed  views  and  actual  practice."  (p.  1 34) 

In  proposing  a  model  for  research  on  teacher  knowledge,  Fennema  and  Franke  (1992)  argued  that  teacher 
knowledge  cannot  be  separated  from  subject  matter,  from  how  that  subject  matter  can  be  represented  for  learners, 
from  what  we  know  about  students'  thinking  in  specific  domains,  or  from  teacher  beliefs.  Fennema  and  Franke 
pointed  out  that  "...within  a  given  context,  teachers'  knowledge  of  content  interacts  with  knowledge  of  pedagogy 
and  students'  cognitions  and  combines  with  beliefs  to  create  a  unique  set  of  knowledge  that  drives  classroom 
behavior."  (p.  162) 

In  discussing  teacher  knowledge,  Berliner  (1986)  argued  that  successful  teachers  develop  expertise  in 
pedagogy  and  content  knowledge,  and  understand  how  these  two  forms  of  knowledge  interact  in  teaching.  In  the 
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context  of  the  current  study,  this  suggests  that  successful  teachers  will  adapt  instructional  strategies  to  match 
relevant  content  factors  (e.g.,  degree  to  which  the  content  is  well-structured). 
Instruction  for  Well-Structured  Content 

Rosenshine  (1986a,  1986b)  argued  that  there  is  ample  research  to  describe  a  pattern  of  instruction  (referred 
to  as  explicit  teaching,  direct  instruction,  systematic  teaching  and/or  active  teaching)  that  is  useful  for  teaching 
any  well-structured  discipline  where  the  objective  is  to  teach  performance  skills  or  mastery  of  a  body  of 
knowledge.  This  pattern  was  thought  to  be  most  important  for  young  learners,  slow  learners,  and  "...for  all 
learners  when  the  material  is  new,  difficult,  or  hierarchical"  (Rosenshine,  1986a,  p.  62)  and  less  relevant "...  in 
areas  which  are  'ill  structured,'  that  is,  where  the  skills  to  be  taught  do  not  follow  explicit  steps."  (Rosen.; nine, 
1986b,  p.  6) 

In  an  extensive  review  of  the  literature,  Rosenshine  and  Stevens  (1986)  examined  several  major  studies,  for 
example,  the  Texas  First  Grade  Reading  Group  Study  (Anderson,  Evertson,  &  Brophy,  1979,  1982)  and  the 
Missouri  Mathematics  Effectiveness  Study  (Good  &  Grouws,  1979).  Based  on  these  and  other  studies, 
Rosenshine  and  Stevens  identified  six  instructional  functions  that  effective  teacheis  employ.  Effective  chers 
conduct  daily  reviews  and  check  previous  work,  present  content  in  an  organized  manner,  guide  student  practice, 
provide  feedback  and  correctives,  provide  opportunities  for  independent  practice,  and  conduct  regular  reviews. 
However,  Rosenshine  and  Stevens  cautioned  that  these  instructional  functions  were  most  appropriate  when 
teaching  weil-slructured  content. 
Contend  Structure  Issues 

Content  structure  is  based  on  notions  of  how  and  if  to-be-learned-content  is  organized.  For  example,  Posner 
and  Strike  (1976)  identified  five  general  approaches  for  organizing  course  content:  world-related  sequence, 
inquiry-related  sequence,  learning-related  sequence,  concept-related  sequence,  and  utilization-related  sequence. 
Posner  and  Strike  indicated  that  these  five  sequences  were  "well  grounded  in  what  have  proven  to  be  useful 
distinctions  in  epistemology."  (p.  667) 

In  another  attempt  to  describe  content  organization,  Reigeluth  and  associates  (Reigeluth,  Merrill,  & 
Wilson,  1979;  Reigeluth,  Merrill,  Wilson,  &  Spiller,  1978)  identified  three  content  structures  -  conceptual, 
theoretical,  and  procedural.  Reigeluth  (1987)  argued  that  "...in  all  the  work  that  has  been  done  on  sequencing, 
elaborations  based  on  concepts,  principles,  and  procedures  are  the  only  three  we  have  found,  although  additional 
ones  may  be  identified  in  the  future."  (p.  249)  The  underlying  assumption  in  these  and  other  organizing 
approaches  is  that  to-be-learned  content  is  sufficiently  well-structured  to  allow  for  organization  to  occur. 
However,  some  would  argue  the  point  that  all  content  is  well-structured. 

In  a  critical  review  of  Reigeluth's  work,  Wilson  and  Cole  (1992)  take  issue  with  the  notion  that  content  can 
(and  should)  be  characterized  within  the  confines  of  any  one  knowledge  structure.  Citing  numerous  sources, 
Wilson  and  Cole  concluded  "...there  is  no  single  right  way  to  represent  knowledge,  even  for  a  given  context  or 
instructional  purpose."  (p.  68)  They  go  on  to  argue  that  "  a  single  representation  of  structure  could  possibly 
limit  students'  personal  constructions  of  meaning  from  the  content."  (p.  69)  While  Wilson  and  Cole  are 
focusing  on  Reigeluth's  work,  they  are  simultaneously  raising  questions  about  the  utility  of  an  instructional 
approach  which  presupposes  the  existence  of  well-structured  content. 

Assuming  that  some  to-be-lcarned  content  is  well-structured  and  other  is  ill-structured  and  that  this 
distinction  has  implications  for  selecting  instructional  strategies,  the  discussion  now  turns  to  a  brief  review  of 
the  teacher  decision  making  literature  in  the  context  of  expert  and  novice  teachers. 
Expert/Novice  Teacher  Decisions 

Considerable  research  has  been  conducted  examining  the  differences  between  expert  and  novice  teachers  in 
terms  of  their  decision  making.  Westerman  (1991)  conducted  a  study  comparing  the  thinking  of  expert  and 
novice  teachers  during  three  stages  of  decision  making:  planning,  teaching,  and  reflecting.  Westerman 
concluded  that  novice  teachers  tend  to  develop  lesson  plans  in  which  content  knowledge  and  theoretical 
knowledge  of  teaching  are  minor  influences. 

Wesierman's  conclusion  was  consistent  with  Borko  and  Livingston  (1989)  who  examined  the  planning, 
teaching,  and  reflections  of  student  teachers  and  their  cooperating  teachers  and  found  differences  between  the  two 
groups.  Specifically  Borko  and  Livingston  found  that  expert  teachers  displayed  more  pedagogical  knowledge, 
content  knowledge,  and  pedagogical  content  knowledge  than  did  novices.  Similarly,  Brown  and  Borko  (1992) 
reviewed  the  literature  on  expert  and  novice  teachers  and  concluded  that  expert  teachers  typically  display  more 
pedagogical  content  knowledge  than  do  novices. 
Literature  Summary 

In  summary,  the  literature  generally  supports  the  notion  that  content  can  be  thought  of  along  a  continuum 
ranging  from  well-structured  to  ill-structured  and  that  instructional  decisions  should,  in  part,  be  based  on  the 
extent  to  which  the  content  is  well-structured.  Furthermore,  the  literature  suggests  that  experienced  teachers  will 
be  more  likely  to  make  instructional  planning  decisions  based  on  this  consideration  than  will  novice  teachers. 
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The  research  described  below  examines  experienced  and  novice  teachers'  instructional  planning  decisions  in  the 
context  of  well-  and  ill-structured  content. 

Method 

Subjects 

As  detailed  in  Tables  1  and  2  below,  a  total  of  152  preservice  (i.e.,  novice)  and  153  in-service  (i.e., 
experienced)  teachers  participated  in  the  study. 

Table  1.  -  Novice  teacher  demographics 

Variable  N  % 


Major 

Early  Childhood  21  13.8% 

Elementary  71  46.7% 

Secondary  54  35-6% 

Other  6  3.9% 

Total  152  100% 


Table  2.  -   Experienced  teacher  demographics. 


Years  Teaching  Experience 


6d0 

LL15 

16-20 

26± 

leal 

Early  Childhood 

7 

2 

0 

0 

0 

0 

9 

Elementary 

23 

13 

10 

7 

3 

0 

56 

Junior  High 

14 

7 

3 

7 

4 

0 

35 

Secondary 

13 

10 

4 

6 

3 

1 

37 

Post  Secondary 

6 

1 

2 

2 

3 

1 

15 

Other 

1 

0 

0 

0 

0 

0 

1 

Total 

64 

33 

19 

22 

13 

2 

153 

Instrument 

Based  on  Rosenshine  and  Steven's  (1986)  work,  a  survey  was  constructed  to  assess  the  likelihood  that 
no\  !ce  and  experienced  teachers  would  include  selected  instructional  functions  in  lessons  dealing  with  well- 
structured  and  ill-structured  content.  The  survey  included  a  brief  introduction  with  explanations  and  examples  of 
well-structured  and  ill-structured  content.  The  survey  consisted  of  nineteen  instructional  strategies.  Participants 
responded  to  each  strategy  in  terms  of  how  likely  it  was  for  that  strategy  to  be  included  in  a  lesson  on  well- 
structured  content  and  then  in  a  lesson  on  ill-structured  content.  A  five  point  Likert-type  scale  was  used  with  a 
response  of  "4"  indicating  "Always"  and  a  response  of  "0"  indicating  "Never."  The  nineteen  items  were 
organized  into  one  of  the  six  instructional  functions  identified  by  Rosenshine  and  Stevens  (see  Table  3). 
Demographic  data  were  also  collected  on  the  survey. 
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Table  3.  -  Instructional  function  clusters  and  related  items  (Roseoshine  &  Stevens,  1986). 

Instructional  Function/Items 


I.  Daily  review  and  checking  previous  work 

1.  Check  homework 

2.  Review  relevant  past  learning 

3.  Review  prerequisite  skills 

II.  Present  new  content/skills 

4.  Provide  statement  of  objectives 

5.  Proceed  in  small  steps  but  at  a  rapid  pace 

6.  Provide  illustrations  and  examples 

III  Guided  student  practice 

7.  Initial  student  practice  takes  place  with  teacher  guidance 

8.  Initial  practice  is  sufficient  so  that  students  can  work  independently 

9.  Prompts  are  provided  during  guided  practice 

10.  Guided  practice  is  continued  until  a  success  rate  of  80%  is  achieved 

IV  Feedback  and  correctives 

11.  Quick  correct  responses  are  followed  by  another  question 

12.  Slow  correct  responses  are  followed  by  helpful  feedback  (Right  because...) 

13.  Students  are  monitored  for  systematic  errors 

V.     Independent  student  practice 

14.  Independent  practice  to  over-learning  (responses  are  firm  and  quick) 

15.  Independent  practice  until  95%  correct  rate  is  achieved 

16.  Students  help  accountable  for  seatwork 

Weekly  and  monthly  reviews 

17.  Systematic  review  of  previously  teamed  material 

18.  Frequent  tests 

19.  Reteaching  of  material  missed  in  tests 


Procedure 

Data  collection  was  accomplished  in  two  ways.  Novice  teachers  completed  the  survey  while  enrolled  in  an 
undergraduate  teacher  education  course.  Experienced  teachers  were  surveyed  by  graduate  students  as  a  part  of  a 
research  methods  course  activity.  No  attempt  was  made  to  randomize  or  stratify  either  the  novice  or  experienced 
teacher  groups  participating  in  the  study. 

Participants  were  told  that  the  survey  was  part  of  a  lesson  planning  study  and  that  their  responses  would  be 
anonymous.  Most  participants  completed  the  survey  in  less  than  ten  minutes.  Data  from  the  completed 
surveys  were  entered  into  a  SPSS-X  database  and  analyzed. 

Means  and  standard  deviations  were  computed  for  each  cluster  by  groups  (novice/experienced)  and  by  content 
structure  (ill/well).  T-tests  were  calculated  on  the  mean  differences  between  scores  for  well-  and  ill-structured 
content,  and  between  scores  for  novice  and  experienced  teachers.  The  results  of  this  analysis  are  summarized  and 
discussed  below. 
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Results 

As  shown  in  Table  5,  novice  teachers'  means  for  lessons  on  well-structured  and  ill-structured  content  ranged 
from  a  low  of  2.40  (Independent  Practice  for  ill-structured  content)  to  a  high  of  3.33  (Daily  Review  &  Check 
Homework  for  well-structured  content).  For  experienced  teachers,  the  means  ranged  from  a  low  of  1 .99  (Weekly 
&  Monthly  Review  for  ill-structured  content)  to  a  high  of  3.20  (Daily  Review  &  Check  Homework  for  well- 
structured  content).  Interestingly,  nov  ;e  teachers'  scores  were  always  higher  than  experienced  teachers'  scores 
regardless  of  instructional  function  or  content  structure.  Data  analysis  (t-test)  indicated  significant  differences 
(p<.001)  in  the  likelihood  that  a  specific  instructional  function  would  be  used  for  teaching  well-  verses  ill- 
structured  content.  This  difference  was  found  for  novice  and  experienced  teachers. 


Table  5.  -  Reported  likelihood  of  using  instructional  functions  with  well-  and  ill-structured  content. 


(4  «=  Always,  0  =  Never) 


Novice  Teachws 

Experienced  Teachers 

Well- 
Structured 

Ill- 
Structured 

Well- 
Structured 

m- 

Structured 

M 

(SD) 

M 

(SD) 

Mean 
Difference 

M 

(SD) 

M 

(SD) 

Mean 

Difference 

Daily  Review  & 
Check  Homework 

3.33 

(-54) 

2.59 

(.85) 

.74* 

3.20 

(.57) 

2.54 

(.72) 

.66* 

Presentation 

3.23 

(.50) 

2.66 

(.75) 

.57* 

3.12 

(.53) 

2.60 

(.77) 

.52* 

Guided  Practice 

3.00 

(.54) 

2.62 

(■73) 

.38* 

2.95 

(.57) 

2.41 

(.79) 

.54* 

Correctives  & 
Feedback 

3.09 

(.55) 

2.47 

(.86) 

.62* 

2.82 

(.56) 

2.28 

(.77) 

.54* 

Independent 
Practice  (Seatwork) 

2.83 

(-63) 

2.40 

(.80) 

.43* 

2.54 

(.68) 

1.97 

(.74) 

.57* 

Weekly  &  MonthI} 
Reviews 

3.19 

(.55) 

2.41 

(.83) 

.78* 

2.73 

(.75) 

1.99 

(.90) 

.74* 

*p<.001 


Somewhat  surprisingly,  there  was  no  significant  difference  between  novice  and  experienced  teachers  in  terms 
of  their  reported  likelihood  of  using  different  instructional  strategies  based  on  content  structure  (see  Table  6). 
Ranging  from  .04  to  .16,  none  of  the  mean  differences  between  experienced  and  novice  teachers  approached 
statistical  significance.  Put  another  way,  both  novice  and  experienced  teachers  indicated  they  would  vary  their 
instructional  strategies  based  on  content  structure  considerations,  and  their  reported  variations  were  similar. 
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Table  6.  -  Means,  standard  deviations  and  mean  differences  of  reported  likelihood  to  use  various 
instructional  strategies  when  teaching  different  types  of  content. 
(4  =  Always,  0  -  Never)  


Novice  Teachers 

Experienced  Teachers 

Mean 

Difference 

Difference  Between 
Well-Structured  & 
Ill-Structured 

Difference  Between 
Well-Structured  & 
Ill-Structured 

Between 

Novice  & 

M 

(SD) 

M 

(SD) 

Daily  Review  & 
Check  Homework 

.74 

(.88) 

.66 

(.75) 

.08 

Presentation 

.57 

(.87) 

.52 

(.75) 

.05 

Guided  Practice 

.38 

(.85) 

.54 

(.74) 

.16 

Correctives  & 
Feedback 

.62 

(.96) 

.54 

(.80) 

.08 

Independent 
Practice  (Seatwork) 

.43 

(.90) 

.57 

(.79) 

.14 

Weekly  &  Monthly 
Reviews 

.78 

(1.02) 

.74 

(.89) 

.04 

*p<.05 

Discussion 

In  this  study,  both  novice  and  experienced  teachers  indicated  they  would  vary  instructional  strategies  based 
on  content  structure.  Furthermore,  the  variations  reported  by  novice  teachers  were  similar  to  those  reported  by 
experienced  teachers.  These  results  have  implications  for  teacher  preparation  in  the  context  of  teacher  knowledge 
and  practice.  The  results  also  have  implications  for  teacher  evaluation.  The  discussion  which  follows  examines 
these  issues. 

Teacher  Knowledge  and  Practice 

As  discussed  earlier  in  the  literature  review,  Berliner  (1986)  argued  that  successful  teachers  develop  expertise 
in  pedagogy,  content  knowledge,  and  their  interaction  as  they  amass  a  large  quantity  of  knowledge  through 
teaching  experience.  Furthermore,  Westerman  (1991),  Borko  and  Livingston  (1989),  and  Brown  and  Borko 
(1992)  pointed  out  that  experienced  teachers  tend  to  display  greater  pedagogical  content  knowledge  which  is 
superior  to  that  of  novice  teachers.  Surprisingly,  in  the  current  study,  novice  teachers  displayed  a  knowledge  of 
the  interaction  of  pedagogy  and  content  structure  in  the  absence  of  extensive  teaching  experience.  One  might 
wonder  then,  how  was  this  knowledge  obtained? 

In  a  study  of  over  400  preservice  teachers  Serow,  Eaker,  &  Forrest  (1994)  found  that  preservice  teachers' 
orientations  to  teaching  "seemed  largely  independent  of  their  exposure  to  teacher-education  curricula". ...and  "were 
typically  grounded  in  personal  experience  rather  than  pedagogical  theory  or  research."  (p.  46)  Serow,  et  al, 
indicated  that  preservice  teachers'  personal  experience  has  been  derived,  in  part,  from  years  of  observing  their 
own  teachers'  pedagogical  practices  suggesting  that  "...few  other  fields  provide  potential  recruits  with  such  close- 
up,  sustained  exposure  to  their  work."  (p.43)  In  the  current  study,  the  novice  teachers  were  likely  making 
instructional  strategy  decisions  based  on  years  of  observing  their  own  teachers,  rather  than  on  knowledge  and 
skills  developed  through  their  teacher  preparation  program  (as  borne  out  by  the  fact  that  no  decision-making 
differences  could  be  attributed  to  years  in  the  teacher  preparation  program). 
Teacher  Evaluation 

While  somewhat  tangential,  the  basis  for  this  study  (specifically  Rosenshine  &  Stevens,  1986)  and  the 
study  findings  have  implications  for  teacher  evaluation.  Many  teacher  evaluation  instruments  currently  in  use 
are  based  on  one  model  of  effective  instruction  and  do  not  reflect  the  need  for  variations  in  instructional 
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functions  based  on  content  structure.  For  example,  the  Texas  Teacher  Appraisal  System  (TTAS)  was  designed 
with  "efforts  to  build  a  practical,  usable  system  that  can  be  applied  fairly  to  teachers  in  all  subjects  and  grade 
levels"  (p.  6,  TTAS  Appraiser/Teacher  Manual,  Texas  Education  Agency,  Austin,  TX).  However,  Lasater  and 
Hawkins  (1993-4)  reported  three  studies  which  focused  on  art  teachers'  and  art  teacher  evaluators'  perceptions  of 
the  appropriateness  of  the  TTAS  for  use  in  the  art  classroom.  The  majority  of  teachers  questioned  the  validity 
of  the  TTAS  for  evaluating  art  teachers.  Lasater  and  Hawkins  found  similar  views  among  special  education 
teachers  who  reported  a  need  for  content-sensitive  evaluation  instruments.  The  argument  here  is  that 
instruments  lacking  this  sensitivity  not  only  are  incomplete,  they  have  the  potential  of  discounting  "correct" 
decision  making  practices.  According  to  the  current  study,  both  experienced  and  novice  teachers  are  likely  to 
vary  their  instructional  strategies  based  on  content  considerations  (i.e.,  well/ill-structured).  Consequently,  a 
teacher  who  is  teaching  ill-structured  content  and  using  appropriate  strategies  may  receive  a  less  favorable 
evaluation  than  would  a  teacher  who  is  using  inappropriate  strategies  if  the  evaluation  instrument  does  not 
reflect  the  distinction  based  on  content  structure. 

Recommendations  and  Conclusion 

The  findings  of  this  study  are  inconsistent  with  much  of  the  literature  on  teacher  decision  making, 
especially  as  it  related  to  novice  and  experienced  teachers.  Several  additional  studies  will  help  resolve  the 
inconsistencies.  First,  the  current  study  did  not  include  asking  the  subjects  how  they  made  their  decisions.  For 
example,  if  a  subject  indicated  that  a  review  of  prerequisite  skills  was  more  likely  to  be  included  in  a  lesson  on 
well-structured  content  that  in  one  on  ill-structured  content,  the  rationale  underlying  the  decision  is  unknown. 
Was  the  decision  based  on  knowledge  of  content  and  pedagogy,  or  was  it  based  on  other  considerations  (e.g.,  the 
time  required  to  review  prerequisite  skills)?  It  is  conceivable  that  a  novice  teacher's  rationale  is  considerably 
different  from  that  of  an  experienced  teacher,  a  distinction  consistent  with  the  novice/expert  literature. 

Second,  in  the  current  study,  the  subjects  were  given  several  examples  of  well-  and  ill-structured  content  and 
asked  to  indicate  how  likely  it  was  for  selected  strategies  to  be  included  in  an  unspecified  lesson  on  each  type  of 
content.  Consequently,  it  is  unknown  if  similar  decisions  would  be  made  if  the  list  of  strategies  was  not 
provided,  a  condition  more  akin  to  teachers'  daily  practices.  Future  studies  might  provide  two  sets  of  outcomes, 
one  based  on  well-structured  content  and  one  based  on  ill-structured  content,  but  not  provide  a  list  of  strategies. 
Here,  novice  and  experienced  subjects  would  be  asked  to  describe  how  they  would  teach  for  each  set  of  outcomes, 
the  intent  being  to  minimize  the  prompting  provided  by  the  list  of  strategies.  Their  responses  could  then  be 
analyzed  using  the  Rosenshine  and  Stevens  (1986)  framework  as  a  guide. 

Finally,  direct  observation  of  instruction  by  novice  and  experienced  teachers  may  help  close  the  gap  between 
the  current  study  and  the  literature.  Such  a  study  might  begin  with  a  survey  of  the  subjects  (either  closed-ended 
as  in  the  current  study  or  open-ended  as  suggested  above).  Following  the  survey,  the  study  would  extend  into 
the  classroom  where  teacher  decisions  are  enacted.  Lessons  on  well-  and  ill-structured  content  could  be  observed 
and  the  teachers  interviewed  following  the  lessons.  The  interview  could  focus  on  key  events  during  the  lessons 
and  the  teachers'  decision  making  associated  with  the  events.  These  data  could  then  be  analyzed  using  the 
Rosenshine  and  Stevens  (1986)  framework  as  a  guide. 

In  conclusion,  while  the  research  outlined  above  will  address  the  gap  between  the  current  study  and  the 
literature,  one  over-arching  question  cannot  be  ignored.  Are  novice  teachers  able  to  make  more  sophisticated 
decisions  than  they  are  given  credit  for  in  teacher  education  programs?  If  that  is  the  case,  then  perhaps  teacher 
education  programs  can  concentrate  more  on  helping  novice  teachers  implement  these  decisions  and  less  on 
building  the  knowledge  base  underlying  these  decisions. 
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Vast  numbers  of  adults  :  i  our  changing  society  need  specialized  training  to  perform  well  on  the 
job.  Yet,  that  training  often  eludes  people  who  live  in  rural  areas,  for  they  often  have  difficulty  finding 
opportunities  to  receive  further  professional  development.  In  an  attempt  to  improve  services  to  rural 
populations,  distance  education  has  become  an  increasingly  popular  form  of  continued  training  to  adult 
audiences.  The  growing  emphasis  on  distance  education  recognizes  the  needs  of  rural  professionals  who 
may  not  have  the  time  or  resources  to  travel  to  more  central  training  sites. 

Like  some  other  professions,  the  increasing  workplace  demands  on  social  workers  require  a  feasible 
solution  to  providing  on-the-job  training  for  those  who  reside  in  both  urban  and  rural  areas.  Social  workers 
face  a  difficult  situation;  they  need  to  know  more  information  to  keep  up  with  changing  policies  and 
provide  quality  services,  while  at  the  same  time  increased  responsibilities  make  it  more  difficult  to  leave  the 
office  for  training  or  education.  While  distance  learning  opportunities  have  multiplied  greatly  in  the  past 
several  years,  the  lack  of  relevant  course  work  and  access  to  the  technology  necessary  to  participate  in  these 
opportunities  may  prohibit  the  wide  use  of  distance  education  for  rural  social  workers. 

There  is  a  critical  need  to  provide  carefully  designed  training  to  meet  the  challenges  rural  social 
workers  face,  yet  limited  resources  for  staff  development  within  the  social  work  agencies  create  a  difficult 
situation  in  meeting  this  training  need.  The  consultation  and  assessment  process  for  planning  staff  training 
becomes  vital  in  this  situation  because  any  training  offered  must  wisely  utilize  the  limited  resources  by 
concentrating  them  on  the  most  essential  needs.  Schoenmaker  (1993)  addresses  this  phenomenon  and 
reveals  a  trend  toward  better  communication  between  trainees  and  training  developers  in  order  to  validate 
design  decisions  prior  to  developing  the  training  materials.  The  training  team  in  this  study  is  practicing 
such  communication. 

Following  a  needs  assessment,  multimediated  training  for  social  work  professionals  was  planned. 
The  first  of  ten  modules  was  designed,  developed,  and  beta  tested  prior  to  field  placement. 

This  paper  reports  results  of  a  beta  test  specifically  aimed  at  checking  the  perceived  quality  of  a 
multimedia  lesson  prior  to  its  release  to  a  field  test  situation.  It  represents  one  step  within  the  process  of 
maintaining  communication  between  the  trainees  and  the  training  developers.  In  this  case,  the  trainees  are 
social  workers  who  live  in  rural  areas  and  the  developers  are  a  team  of  people  with  various  professional 
skills. 

BACKGROUND 

Collaboration  between  two  major  institutions  in  a  rural  midwestern  state  identified  specific 
training  needs  for  child  welfare  workers  and  a  five-year  plan  to  meet  them.  The  State  Department  of  Social 
and  Rehabilitation  Services  (SRS)  contracted  with  the  a  major  state  university  to  develop  a  series  often 
multimedia  lessons  (modules)  for  child  welfare  workers  that  would  emphasize  the  educational  needs  of 
workers  in  the  rural  parts  of  the  state. 

The  first  step  of  the  learning  process  assessed  the  workers'  perceptions  of  their  needs.  The  second 
step  developed  a  plan  to  meet  those  needs.  The  third  step  involved  design  and  development  of  educational 
modules  using  multimedia  (videodisc,  computer,  workbook)  for  the  delivery.  This  paper  reports  on  the 
beta  test  phase  of  the  first  module  of  the  project,  which  was  intended  to  help  guide  the  project  toward 
improvement  prior  to  releasing  the  module  as  a  finished  product.  The  beta  test  investigated  user  opinions 
about  the  content,  instructional  design,  technical  integrity,  ease  of  use,  and  general  quality  of  the  module. 
Because  people's  opinions  about  self-efficacy,  ease  of  use,  and  quality  can  influence  their  attitudes  toward 
the  instruction  and  in  turn,  influence  performance  and  implementation  patterns  (Fullan,  1982;  Klein, 
Knupfer,  &  Crooks,  1993;  Rogers,  1983;  Soltani,  1995),  it  is  important  that  designers  consider  learners' 
opinions  about  the  quality  of  material  in  terms  of  its  usability  and  perceived  value. 

This  program  was  designed  so  learners  could  set  the  pace  of  the  lesson,  and  work  through  it  at  a 
rate  commensurate  ,/ith  their  available  time,  ability,  and  motivation.  The  learners  have  the  option  to  start 
and  stop  at  any  time,  and  to  repeat  portions  when  necessary  or  desired. 

The  underlying  rationale  for  self-pacing  is  that  individuals  learn  at  different  rates.  One  claim  made 
in  favor  of  using  technology  for  instructional  is  the  ability  of  technology  to  adapt  instruction  to  individual 
differences  thereby  gaining  efficiency  while  at  the  same  time  promoting  achievement  Semb  et  al  (1993). 


Research  clearly  demonstrates  that  a  variety  of  self-paced  formats  such  as  computer-based 
instruction  (Bangert-Drowns,  Kulik,  &  Kulik,  1985;  Kulik  &  Kulik,  1987,  1989;  Kulik  &  Bangert- 
Drowns,  1985;  Orlansky  &  String,  1981),  Bloom's  Learning  for  Mastery  (Kulik,  Kulik  &  Bangert-Drowns, 
1989),  and  Keller's  Personalized  Systems  of  Instruction  (Kulik,  Jaska,  &  Kulik,  1978)  engender  more 
positive  attitudes  about  learning  than  conventional  instruction.  In  addition,  they  are  more  effective, 
efficient,  and  preferred  by  the  learners.  (Soltani,  1995) 

Formative  Evaluation 

It  is  helpful  to  think  of  formative  evaluation  as  an  explicit  stance  throughout  the  design  process  in 
which  one  collects  data  and  uses  them  to  inform  the  design  procedures.  While  the  project  is  still  fluid,  the 
beta  test  serves  as  part  of  the  formative  evaluation.  The  results  of  the  beta  test  will  be  applied  to  design 
adjustments  in  this  first  module  and  the  remaining  nine  modules. 

Several  researchers  have  deveVpcd  a  basic  set  of  categories  by  which  instructional  software  can  be 
evaluated.  For  example,  Morrison,  (1987)  placed  maximum  emphasis  on  the  facet  of  interactivity  which 
he  defined  as  'the  learner  in  conversation  with  himself  over  the  material  to  be  learned'.  He  addressed  the  idea 
of  adaptability  where  the  program  gives  control  back  to  the  learner.  Morrison  claimed  that  well-designed 
products  offer  users  an  opportunity  to  find  individual  pathways  through  material  and  encourage  learners  to 
develop  higher  cognitive  skills  of  self-assessment  and  evaluation  of  their  own  learning  style  and  patterns. 

Plowman(1989)  explored  such  facets  of  learning  design  in  as  it  specifically  applies  to  interactive 
video.  Important  categories  for  evaluation  which  emerged  include  considerations  of  navigation  and  mobility 
through  the  software,  physical  aspects  of  human-computer  interaction,  screen  presentation,  quality  of  learner 
interactions,  consistency  of  appearance  and  functionality,  visual  clarity,  legibility,  access  to  support  in  the 
form  of  'help'  and  'next'  options,  availability  of  a  bookmark  facility,  user  support  tools  like  a  glossary  or 
other  reference  items,  the  style  of  user  interface  which  might  embody  windows  or  other  devices  to  maximize 
the  amount  of  easily  available  information.  The  design  of  this  project  considered  all  of  those  factors,  plus 
some  additional  ones. 

Instructional  Courseware  Design 

A  courseware  production  system  has  been  defined  as  the  entire  set  of  activities  beginning  with 
vague  ideas  or  concepts  to  be  taught  and  ending  with  Finished  materials  for  the  users  (Karrer,  1987).  A 
production  strategy  means  the  combination  of  resources  to  set  up  and  run  a  production  process  of 
courseware  development;  it  must  address  the  questions  of  manpower,  required  skills,  and  tools  for 
development  (Karrer,  1987). 

Many  authors  have  called  for  a  team  approach  to  courseware  development.  Ally  (1985)  suggested  a 
five-member  courseware  design  team  which  included  a  manager,  instructional  designer,  computer  specialist, 
communication  specialist,  and  content  specialist.  Collis  and  Gore  (1987)  called  for  a  collaborative  model 
which  included  in  its  design  a  rigorous  system  which  enabled  team  members  to  check  and  re-check  work 
done  by  themselves  and  others  by  assessing  congruency  at  several  points  in  the  development  process. 
Faiola  (1989)  described  the  traditional  team  with  one  more  person,  the  graphic  or  visual  specialist.  Because 
assessment  is  such  an  important  part  of  the  process  Collis  and  Gore  suggest  placing  in  the  collaborative 
model  a  step  for  field  testing  prototypes. 

Because  of  the  specific  nature  of  the  profession,  social  workers  can  have  a  tremendous  impact  upon 
human  lives  at  particularly  vulnerable  times.  Therefore,  it  is  important  that  they  receive  ongoing  education 
that  will  enhance  their  practical,  applied  skill  concerning  human  services.  Reay  (1986)  argues  in  favor  of 
changing  the  focus  of  social  work  training  from  content  issues  to  process  issues.  This  could  be  done  by 
limiting  the  theoretical  information  and  choices,  and  rigorously  encouraging  students  to  master  conditioned 
response  (CR)  learning  and  progress  to  understanding  the  learning  process  and  applying  the  theory.  Using 
this  approach  of  embedding  theory  in  practice,  presents  on  an  opportunity  for  the  combination  of  modern 
technology  and  adult  methods  of  learning  to  help  students  master  the  process  of  systematically  applying 
theory  to  practice. 

General  psychological  research  and  theory  yield  fundamental,  widely  generalized  principles 
concerning  how  individuals  think  and  learn.  General  psychological  principles  are  largely  media-independent. 
Psychological  constructs  such  as  schemata  (Norman,  Gentner,  &  Stevens,  1976;  Spiro  &  Anderson,  1981); 
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meaning  (Bandura,  1971);  elaboration  (Wagner  &  Rohwer,  1981);  and  situated  cognition  (Brown  &  Duguid, 
1 993;  Suchman,  1 99 1 )  emphasize  how  individuals  organize  and  retrieve  knowledge  and  establish  meaning, 
and  hence  have  far  reaching  implications  for  the  design  of  learning  systems  that  will  allow  the  application 
of  theory  to  practice. 

Motivational  Adult  Instruction 

Designing  instruction  for  adult  learners  must  take  in  those  special  considerations  that  are  a  major 
factor  in  adult  learning.  Adults  have  specific  and  unique  motivational  needs.  The  motivational  aspect  of  the 
design  can  increase  the  learners  efforts  and  attract  learners  to  the  instructional  content  and  methods.  (Keller, 
1983) . 

The  motivational  instruction  can  increase  the  amount  of  time  and  effort  the  learner  will  put  into 
the  task.  Keller  (1987),  Keller  and  Suzuki  (1988)  as  well  as  Keller  and  Kopp  (1987)  identify  four  categories 
of  motivating  strategies  in  learning  situations,  which  include:  attention,  relevance  (Keller's  ARCS  Model), 
confidence,  and  satisfaction.  All  of  these  strategies  were  incorporated  into  the  design  of  this  program.  The 
needs  of  the  learner  were  assessed  and  a  design  team  addressed  each  aspect  as  follows:. 

Attention.  Content  related  anecdotes,  case  studies,  and  biographies  are  appropriate  along  with 
animation  and  sound.  (Keller,  1987)  The  specific  content  developed  by  the  subject  matter  experts  contains 
all  the  current  topics  and  issues  pertinent  to  the  social  workers.  The  appropriate  designs  by  the  graphic 
artist  take  into  consideration  what  is  appropriate  for  the  user/client 

Relevance.  Showing  how  the  instructional  materials  relate  to  specific  and  current  needs.  How 
the  instruction  and  information  relates  to  ones  job,  and  to  real  life. 

Confidence.  Giving  the  learner  the  locus  of  control.  Throughout  the  program,  the  user  has 
options,  the  ability  to  make  choices  and  the  freedom  of  navigation. 

Satisfaction.  Using  skills  learned  in  simulated  situations  Keller  &  Kopp  (1987).  The 
interactive  video  component  takes  the  learner  into  environments  which  in  effect  lets  you  think  while  seeing 
and  doing. 

Adult  professionals  typically  need  to  improve  work  performance.  The  computer-based  interactive 
videodisc  training  program,  has  the  potential  to  assist  in  learning  and  thus  is  very  motivational. 

Continuing  Education 

Computer-based  simulations,  especially  those  incorporating  interactive  video  capabilities,  have 
been  demonstrated  as  effective  instructional  methods  in  medical,  industrial,  military  and  business  context 
(Reeves,  1988).  One  positive  aspect  concerning  this  formative  evaluation  is  that  the  module  was 
previously  approved  for  three  continuing  education  credits. 

Individualized  Learning  Process 

According  to  Gotz,  (1990),  the  learner  can  designate  the  time  and  place  of  learning.  Advanced 
programs  offer  different  paths  of  instruction  in  the  processing  of  instructional  material.  The  amount  of 
material  can  be  selected  according  to  the  skills  and  qualifications  of  the  learner.  The  learner  can  call  up  help 
programs  and  systematically  prepare  for  tests  through  simulations  and  visual  instruction.  A  knowledge 
control  of  or  classification  of  tests  results  can  be  integrated  into  the  program  and  the  learning  sequences  can 
be  repeated  as  often  as  needed. 

Distance  Education 

Distance  Education,  although  long  established,  is  currently  in  a  stage  of  rapid  evolution.  The 
most  fundamental  motivation  for  distance  education,  that  of  reaching  learners  who  are  geographically 
hindered  from  meeting  in  a  traditional  setting,  has  been  expanded  to  include  motivations  such  as  time 
tailoring  and  flexibility  of  learner  characteristics.  At  the  same  time,  advances  in  the  technological  media  are 
adding  new  dimensions  to  distance  education.  In  particular,  the  contribution  of 'electronic  distance 
education'  strategies  with  respect  to  the  problems  of  inequities  in  opportunity  for  isolated  learners  or 
learners  with  special  needs  is  becoming  increasingly  recognized  (Stubbs  &  Burnham,  1990). 
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Social  and  Rehabilitative  Services  was  especially  supportive  of  the  project  because  of  the  large 
number  of  social  workers  from  rural  areas  that  would  be  trained..  They  need  to  receive  comparable  services 
and  have  equity  of  access.  The  computer-based  interactive  videodisc  program  is  meeting  that  need. 

Mecklenburger(1990)  stated  that  an  advan.'age  of  multimedia  and  the  capabilities  it  adds  to  distance 
learning  is  that  learners  can  learn  wherever  they  are.  One  of  the  advantages  of  this  is  anytime,  anywhere 
learning. 

Effects  of  Multimedia 

The  popularity  of  electronic  learning,  especially  interactive  multimedia,  centers  around  the 
potentials  of  the  media.  Kelly  (1990)  cited  four  such  potential: 

1 .  The  new  media  provides  rich  opportunities  to  learn.  The  combination  of  sight,  sound,  and  interactivity 

provides  opportunities  to  learn  new  things  without  ever  leaving  the  classroom. 

2.  The  media  also  provides  the  unexpected  benefit  of  increasing  opportunities  for  teaching  people  to  work 

in  teams  and  honing  other  interpeisonal  skills. 

3.  The  technologies  can  make  it  easier  to  meet  the  varying  needs  of  individual  learners  by  being  able  to  : 

diagnose  what  the  learner  knows;  identify  any  learning  difficulties  the  learner  might  have;  and  then 
attend  to  those  needs.  Also,  the  technologies  enable  the  learners  to  go  at  their  own  pace. 

4.  The  new  technologies  can  provide  new  tools  to  assessment  of  skills. 

Research  provides  ample  evidence  for  the  power  of  visuals  and  interactive  multimedia  to  enhance 
learning  (Carlson  &  Falk  1990-91 ;  Dwyer  1982.  1978;  Knupfer  &  Clark,  1992).  This  multimedia  module 
had  the  potential  to  offer  visually  rich  instruction. 

RESEARCH  QUESTIONS 

Several  research  questions  were  of  interest.  The  research  questions  of  particular  interest  in  this 
formative  evaluation  were: 

1 .  Do  learners  like  the  module? 

2.  Do  learners  believe  that  the  lesson  is  realistic,  valuable,  and  relevant? 

3.  Do  learners  believe  that  the  multimedia  format  works  well  for  delivery  of  this  lesson? 

4.  Do  learners  believe  the  video  sequences  are  valuable? 

5.  Can  learners  navigate  easily  within  the  lesson? 

6.  Are  learners  likely  to  use  the  extra  features,  such  as  the  glossary  or  content  map? 

7.  Does  the  module  function  properly,  including  the  bookmark  feature? 

8.  What  strengths,  weaknesses,  and  suggestions  do  the  learners  have? 

9  How  much  time  does  it  take  to  complete  the  module? 

10.  Are  there  differences  in  the  way  social  workers  and  instructional  designers  rate  the  module? 

METHODOLOGY 

Subjects 

Subjects  for  this  study  included  twenty  people;  1 1  were  social  work  graduate  students  or 
professionals,  and  9  were  graduate  students  of  educational  technology.  The  social  work  subjects  were 
selected  because  they  would  be  typical  of  the  group  of  learners  for  whom  the  instruction  was  designed. 
They  likely  would  represent  about  the  same  technical  skill  level  and  would  have  background  knowledge 
necessary  to  judge  the  merit  of  the  content  and  presentation  style  of  the  training.  The  educational 
technology  students  were  typical  of  instructional  designers  who  would  be  responsible  for  developing  similar 
multimediated  lessons. 


Materials 

Subjects  in  this  study  used  the  multimediated  lesson  titled  "Child  Development"  which  is  the  first 
module  of  the  Building  Family  Foundations  series.  In  addition,  they  completed  a  43-item  questionnaire. 

The  multimediated  lesson  was  based  on  an  IBM  486  computer  system  with  a  Pioneer  videodisc 
player.  It  used  Toolbook  as  the  authoring  system. 

A  set  of  written  materials  composed  a  workbook  that  accompanied  the  electronic  part  of  the  lesson. 
The  workbook  was  designed  as  a  reference  that  social  workers  could  keep  close-at-hand  and  xerox  as  needed 
for  doing  their  job  independently  or  working  with  clients.  The  workbook  contained  instructions  for  starting 
the  lesson,  and  any  text-based  information  that  a  social  worker  might  like  quick  access  to,  especially  when 
working  with  a  client.  For  example,  the  workbook  contained  a  glossary  of  terms,  informational  articles, 
references  that  a  social  work  might  like  to  refer  to,  and  forms  that  could  be  duplicated  and  used  on  the  job  or 
distributed  to  clients. 

The  survey  instrument  used  to  collect  the  data  contained  29  items  based  upon  a  four-point  Likert- 
type  scale,  10  open-ended  questions,  and  4  rank-order  questions  asking  what  mode  of  delivery  subjects  would 
prefer  for  future  lessons. 

Procedure 

Each  subject  was  given  a  brief  introduction  to  the  equipment  and  asked  to  complete  a  learning 
module  independently  and  in  its  entirety.  The  instructions  included  a  request  for  each  subject  to  try  to  use 
the  bookmark  feature  one  time;  other  than  that,  no  specific  requests  were  made  concerning  the  process  of 
module  completion. 

Immediately  after  completing  the  module,  each  subject  filled  out  a  questionnaire  about  the 
experience.  The  completed  questionnaire  was  collected  before  the  subject  left  the  sitt.  Summary  statistics 
were  gathered  and  changes  recommended  based  upon  the  beta  test. 


Data  Analysis 

The  data  were  analyzed  using  summary  statistics,  chi  square  analysis,  and  qualitative  analysis.  The 
structured  questions  combined  with  the  open-ended  questions  provided  an  opportunity  to  gather  the  kinds  of 
information  that  we  anticipated  was  important  as  well  as  the  spontaneous  reactions  of  the  subjects. 

RESULTS  AND  DISCUSSION 

The  subjects  who  used  the  multimediated  lesson  represented  two  occupations  and  thus,  two  points 
of  view;  those  knowledgeable  in  the  social  work  content  and  those  knowledgeable  about  interactive 
courseware  design.  The  social  work  subjects  (SW)  were  likely  to  provide  information  about  such  things  as 
content  accuracy,  flow,  relevancy,  and  presentation  style,  while  the  instructional  design  (CD)  subjects  were 
likely  to  react  to  such  things  as  human  interface  design  principles,  graphic  design  and  use,  navigation,  and 
other  features  related  to  instructional  design. 

One  potential  area  of  difficulty  was  experience  with  this  type  of  mediated  instruction.  Of  the  1 1 
SW  subjects,  4  had  used  a  multimediated  lesson  before  while  7  had  not,  and  of  the  9  ID  subjects,  8  had  used 
a  multimediated  lesson  before  while  1  had  seen  multimedia  but  had  not  actually  experienced  using  it.  Ages 
ranged  from  21  through  47  years,  with  an  average  SW  age  of  27,  an  average  ID  age  of  37,  and  an  average  of 
3 1  years  overall. 

Realism,  Valuable,  and  Format 

The  overall  reaction  of  both  groups  was  that  this  kind  of  program  is  valuable  for  learning  this 
type  of  content  and  it  is  especially  useful  in  the  case  of  a  distant  audience,  such  as  rural  social  workers.  The 
SW  subjects  were  relatively  new  users  of  technology  and  their  reactions  displayed  overall  enthusiasm  of 
having  a  well-organized  instructional  program  that  offered  them  audio,  video,  dynamic  graphics,  and  text. 
Approximately  90  percent  of  the  SW  subjects  and  100  percent  of  the  ID  subjects  agreed  or  strongly  agreed 
that  the  content  was  accurate,  while  100  percent  SW  and  100  percent  ID  subjects  agreed  or  strongly  agreed 
that  the  content  was  realistic. 
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Concerning  relevancy,  82  percent  of  the  SW  subjects  and  78  percent  of  the  ID  subjects  believed 
that  the  content  was  relevant  to  their  needs,  while  9j     .cent  of  the  SW  and  89  percent  of  the  ID  subjects 
believed  that  the  multimedia  format  presented  a  convenient  way  to  learn  the  material.  Even  though  the  ID 
subjects  were  not  studying  social  work,  their  notation  of  relevancy  would  reflect  the  generic  nature  of 
interest  in  child  development.  The  module  was  designed  to  help  social  workers  recognize  children  who 
should  be  referred  to  a  specialist  for  evaluation  and  potential  help.  Certainly  any  parent  or  person  who 
works  with  children  in  any  way  would  have  some  interest  in  the  topic. 

Thirty-six  percent  of  the  SW  group  and  72  percent  ID  group  believed  the  content  needed  more 
depth,  while  91  percent  SW  and  100  percent  ID  agreed  or  strongly  agreed  that  the  module  content  was  clear. 
Virtually  all  of  the  subjects  b  ieved  that  the  lesson  contained  valuable  information  and  that  it  was  valuable 
learning  experience  for  them  to  use  this  form  of  instruction. 

All  of  the  SW  subjects  and  90  percent  of  the  ID  subjects  believed  that  this  style  of  lesson  was  a 
good  way  to  learn  the  content,  while  72  percent  SW  and  87  percent  ID  were  satisfied  with  their  own 
performance. 

Video 

One  area  of  potential  strength  for  this  lesson  was  the  video  component.  Approximately  80  percent 
SW  and  66  percent  ID  agreed  or  strongly  agreed  that  the  video  information  helped  them  to  understand  to 
concepts.  None  of  the  SW  subjects  believed  that  the  video  was  not  relevant,  but  22  percent  of  the  ID  agreed 
that  the  video  was  not  relevant  to  the  lesson  content.  The  ID  subjects  seemed  to  look  more  intensely  at  the 
actual  design  and  expressed  some  concerns  including  a  more  critical  evaluation  of  the  video  portions  of  the 
program. 

The  length  of  the  video  segments  in  relationship  to  the  information  given  and  the  relevancy  of 
specific  visuals  were  critical  points.  Subjects  expressed  a  desire  to  stop  the  video  and  replay  only  small 
sections  rather  than  longer  pieces.  This  suggests  a  need  to  make  sure  there  is  a  good  video  match,  but  even 
further  that  shorter  segments  of  video  be  used  so  that  learners  can  repeat  targeted  sections. 

Navigation,  Functionality 

Navigating  through  the  software  was  a  potentially  problematic  area  for  SW  subjects  who  had  little 
experience  with  this  type  of  mediated  instruction.  Ninety  percent  of  the  SW  subjects  and  100  percent  of  the 
ID  subjects  agreed  or  strongly  agreed  that  the  module  content  was  easy  while  no  one  thought  it  was  difficult 
or  hard  to  use.  Despite  that  fact  that  no  one  thought  is  was  difficult,  9  percent  of  the  SW  subjects  and  22 
percent  of  the  ID  subjects  agreed  or  strongly  agreed  that  the  content  was  confusing. 

Even  though  the  design  team  advocates  users  choice,  there  are  some  controls  on  navigation  to 
assist  with  directing  the  learners  to  prerequisite  information  or  the  desire  of  the  subject  matter  expert  (SME) 
to  be  sure  that  the  learners  cover  a  certain  area  within  a  segment.  The  ID  subjects  saw  this  as  restrictive, 
but  made  the  comment  that  this  may  be  a  good  design  feature  for  new  users  to  help  prevent  them  from 
getting  lost  within  the  program.  The  beta  test  revealed  need  to  make  sure  all  check  marks  stayed  in  place 
for  the  duration  of  a  learner's  use  of  a  module.  The  check  marks  indicate  topics  and  subtopics  that  have 
been  completed.  Learners  wanted  to  be  able  to  back  track  one  screen  at  a  time  to  recherk  information  rather 
than  going  back  to  the  beginning  of  a  section. 

The  bookmark  feature  was  designed  as  an  easy  way  to  restart  the  lesson  after  taking  a  break...  It 
kept  information  such  as  check  marks  to  record  progress  and  the  specific  place  where  a  user  left  off  during 
the  last  usage.  Learners'  individual  records  were  stored  on  their  own  floppy  disk  upon  exiting  form  the 
program.  Among  the  information  tracked  is  length  of  time  spent  on  the  module;  that  is  important  to 
enable  learners  to  get  continuing  education  credit. 

Although  hot  words  with  definitions  in  pop-up  boxes,  an  on-line  glossary,  and  content  map  to 
check  where  they  were  and  where  they  could  hop  to  were  included,  but  they  were  very  virtually  unused.  All 
subjects  tended  to  go  straight  through  the  program  in  a  linear  sequence.  The  bookmark  feature  was 
something  that  might  be  used  in  a  real  work  setting,  but  here  each  person  only  used  it  as  part  of  the  test. 
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The  suggestions  gathered  led  to  changes  in  content,  screen  design,  and  general  flow  of  the  program. 
It  will  be  important  to  give  some  sort  of  orientation  to  the  next  group  so  that  they  know  how  specific 
features  can  help  them.  To  help  clarify  the  lesson  and  alleviate  confusion,  revisions  to  the  content 
introductions,  transitions,  areas  of  emphasis,  and  content  map  were  made. 

Time 

The  average  length  of  time  in  minutes  to  complete  the  module  for  SW  subjects  was  138  (low  50, 
high  230,  SD  48.95)  and  for  ID  subjects  was  142  (low  90,  high  180,  SD  30.32).  Approximately  80 
percent  SW  and  55  percent  ID  believed  that  the  lesson  was  about  the  right  length,  while  9  percent  SW  and 
1 1  percent  ID  believed  that  it  was  boring. 

Strengths 

The  overall  reaction  about  the  strengths  of  the  lesson  referred  to  the  power  of  the  video  to  help 
visualize  concepts  and  the  good  organization  and  logical  flow  of>ontent  categories.  The  aural,  visual,  rtnd 
textual  interaction  with  the  learner  was  viewed  as  an  appropriate  way  to  accommodate  the  different  learning 
styles. 

The  SW  subjects  were  particularly  pleased  with  the  opportunity  to  see  actual  vignettes  of 
situations  that  related  directly  to  the  content,  both  SW  and  ID  subjects  believed  the  navigation  was 
friendly  enough  to  help  alleviate  some  potential  confusion  about  certain  areas  of  the  content. 

Weaknciies 

The  main  weaknesses  of  the  lesson  concerned  the  relationship  of  review  and  test  questions  to 
content  presented,  the  relevancy  and  length  of  video  segments,  and  the  inability  to  go  back  a  few  screens 
without  taking  a  large  jump  backwards. 

Although  the  instructional  designer  had  worked  with  the  SME  to  write  appropriate  objectives  and 
good  questions  matched  to  the  objectives,  the  content  presentation  fell  short  on  emphasizing  the  important 
points.  In  fact  some  questions  contradicted  the  content  and  therefore  were  confusing.  Revisions  to  the 
lesson  presentation  were  necessary  to  provide  enough  clear  and  consistent  information  related  to  the  test 
questions. 

Colors  and  Graphics 

Reactions  to  colors  and  graphics  are  most  likely  related  to  the  learner's  experience.  The  SW 
subjects  like  the  way  that  the  graphics  were  presented  and  the  colors  that  were  used,  while  the  ID  subjects 
wanted  the  graphics  to  be  more  lively  and  found  the  colors  dull.  The  two  groups  had  opposite  views  here, 
reflecting  their  experience  and  expectations. 

CONCLUSIONS 

In  general,  the  subjects  responded  favorably  to  the  multimedia  training  and  believed  that  the  video 
component  added  value  to  the  experience  through  the  use  of  examples  and  demonstrations.  A  few  SW 
subjects  expressed  the  preference  for  traditional  classroom  instruction  where  they  could  ask  questions.  The 
more  experienced  social  workers  made  suggestions  to  improve  the  content,  while  the  less  experienced 
students  and  instructional  designers  focused  their  comments  more  on  the  technology  and  program  flow. 

Although  the  design  team  had  built  some  fancy  features  into  the  module,  the  subjects  did  not  use 
most  of  them.  That  could  have  been  due  to  their  limited  experience  with  multimedia  and  technology  in 
general,  a  way  of  presenting  the  features  that  is  less  than  obvious,  or  lack  of  time  to  explore. 

The  design  adjustments  recommended  from  the  beta  test  can  be  categorized  into  three  areas: 
content,  instructional  design,  and  technical  integrity.  With  the  subject  matter  expert  being  a  member  of  the 
design  team,  the  content  and  clarity  was  one  of  the  positive  features.  In  addition  to  attending  to  the  features 
noted  earlier,  we  recommend  chunking  the  information  better  and  adding  even  more  variety  of  perspectives 
to  cover  a  diverse  population  of  learners.  We  further  recommend  the  inclusion  of  a  person  within  the  design 
team  who  could  be  classified  as  an  expert  who  can  authenticate  certain  ideas  from  within  a  cultural 
perspective.  In  this  case,  the  social  workers  were  very  helpful  as  part  of  the  beta  test  team.  Experts  on 
child  development  and  special  education  were  also  included. 
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The  instructional  design  features  mentioned  earlier  were  corrected  and  the  actual  design  interface 
was  revised  to  improve  navigation.  Changes  were  made  in  fonts,  colors,  layout  of  the  screen.  One 
drawback  of  the  SMEs  being  the  project  directors  has  been  pressure  to  present  information  in  a  way  that  the 
designers  felt  matched  linear  video,  but  not  necessarily  multimedia.  Those  issues  were  revealed  and 
addressed  with  each  member  of  the  design  team  contributing  a  strength. 

The  decision  was  made  to  revise  the  program  that  records  the  learner's  path  throughout  the 
program.  The  bookmark  feature  was  modified  and  improved  to  allow  more  flexibility.  It  is  also  important 
to  pinpoint  a  the  specific  features  of  hardware  that  the  learners  will  have  in  their  various  remote  sites 
because  a  few  of  the  keyboard  variations  can  make  difference  in  program  functionality.  The  programmer 
made  changes  to  avoid  sensitive  features  that  can  be  anticipated,  but  this  limits  our  ability  to  be  specific 
with  learner  instructions. 

We  recommend  that  the  module  be  field  tested  in  a  variety  of  sites  prior  to  full  distribution. 
Within  the  field  test  we  recommend  that  the  data  collected  include  measures  of  attitude,  confidence, 
expectancy,  comfort  level  with  technology,  knowledge,  achievement,  and  satisfaction  with  the  multimedia 
lesson. 
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BUILDING  COMMUNITY: 
DEMOCRACY,  DISCOURSE,  AND  PRAXIS 

There  are  two  positions  I  want  to  take  regarding  this  symposium: 

1 .  As  a  society  we  lack  a  conceptual  framework  to  engage  in  moral  discourse,  and  we  lack  clear 
understanding  of  the  notion  of  democracy;  and, 

2.  To  build  "real"  community  (as  opposed  to  "virtual"  community)  requires  a  means  to  engage 
each  other  in  moral  discourse;  a  clear  understanding  of  the  notion  of  democracy;  and  a  radical  and 
critical  praxis. 

I  have  ten  minutes  in  which  to  establish  these  positions.  Consequently,  these  remarks  will  be  broad,  not 
detailed.  These  remarks  are  meant  to  engage  us  in  conversation.  I  am  not  playing  "devil's  advocate"  here, 
rather  I  am  looking  for  conversation  that  will  push  us  into  areas  that  we  can  explore  toward  building 
community.  My  interest  is  in  building  community  where  we  are.  This  is  particularly  true  for  me  as  an 
educator  within  an  institutional  setting  that  at  times  -.eems  to  be  at  odds  with  building  a  democratic, 
participatory  community. 

These  two  positions  are  inter-related  so  I  will  develop  them  together.  My  discussion  should 
include  the  following  points:  establishing  the  need  for  a  moral  discourse;  presenting  initial  ideas  for  an 
understanding  of  democracy;  and  offering  some  insight  into  the  notion  of  radical  and  critical  praxis  that  will 
provide  the  possibility  for  building  community. 

Traditionally,  schools  have  been  involved/concerned  with  the  notion  of  democratic  ideals  and 
practices.  Thomas  Jefferson  had  specific  proposals  for  schools  in  Virginia;  Horace  Mann  proposed  specific 
reform  through  common  schools;  more  recently,  "A  Nation  at  Risk"  (1983)  proposed  reform  measures 
because  the  schools  were  failing  in  their  mission  of  fostering  strong  "products"  to  maintain  the  economy 
and  the  "American- Way  of  Life". 

Historically,  the  notion  of  democracy  has  changed  over  the  years.  How  we  talk  about  democracy 
has  changed.  School  responses  to  these  changing  notions  of  democracy  have  created  different  programs  of 
reform,  e.g.,  desegregation  of  schools,  Head  Start,  regaining  "lost  economic  and  military  ground",  (see 
Beyer,  1988).  The  shifting  notions  of  democracy  result  in  multiple  responses  that  are  sometimes 
contradictory.  There  is  no  one  vision  of  democracy;  visions  of  democracy  have  "competing  interests"  as 
well  as  differing  notions  of  "social  justice";  and  because  of  our  inability  to  engage  in  moral  dialogue  within 
society  (especially  within  schools),  it  is  increasingly  difficult  to  clarify,  debate  or  act  on  these  visions, 
much  less  develop  historical  understanding.  This  inability  of  society  (and  the  "schools")  to  provide  a  forum 
for  the  open  discussion  of  these  visions  of  democratic  ideas  results  from 

...a  loss  of  communities  within  which  such  discourse  can       become  meaningful  and  prompt  the 
requisite  social  action.  Surrounded  by  larger  institutional  structures  favoring  technization,  commodification, 
and  the  therapeutic  privatization  of  social  relations,  we  have  all  but  lost  a  sense  of  the  collective  social  good 
so  necessary  for  discussion  of  democratic  ideas  (Beyer,  1988,  p.221). 

Beyer  (1988)  argues  that  we  have  "lost  the  ability  to  use  moral  language  sensibly".  Quoting 
Alasdair  Maclntyre: 

what  we  possess  ...  are  the  fragments  of  a  conceptual  scheme,  parts  which  now  lack  those  contexts 
from  which  their  significance  derived.  We  possess  indeed  simualcra  of  morality,  we  continue  to 
use  many  of  the  key  expressions.  But  we  have-  very  largely,  if  not  entirely-  lost  our 
comprehension,  both  theoretical  and  practical,  of  morality  (Beyer,  1988,  p. 223). 

Beyer  (1988)  attributes  the  inability  to  use  moral  language  sensibly  to  forms  of  academic  theorizing  that 
have  "weakened  the  authenticity  and  potency  of  moral  discourse"  (p. 223).  He  discusses  three  such  forms: 
First,  conceptual  analysis  in  philosophy  which  tends  to  separate  moral  debate  from  social  context.  (1 ) 


Secondly,  a  position  of  "epistemological  and  ontological"  dualism  with  a  commitment  to  certitude  through 
forms  of  empirical  investigation.  The  dualism  "inherent  in  positivistic  endeavors  demeans  the  viability  of 
moral  judgments"  (p.225).(2)  And,  third,  the  university  research  community,  as  a  community  of  scholars 
involved  in  the  "conservation,  development,  and  dissemination"  of  our  cultural  heritage,  shifted  their 
allegiance,  becoming  more  closely  allied  with  "social  and  governmental  agencies,  and  responsive  to  the 
demands  of  the  growing  corporate  sector".  This  had  the  effect  of  further  separating  moral  discourse  from 
"the  production  of  knowledge",  which  production  happened  within  increasingly  narrow  disciplines  and  sub- 
disciplines,  while  moral  inquiry  belonged  to  one  area  of  academic  inquiry  (p.225). 

Purpel  (1993)  sees  the  need  for  a  moral  discourse  within  society  generally,  but  within  education 
specifically.  He  identifies  a  dangerous  chasm  between  "mainstream  educational  discourse  and  the  urgent 
social,  political,  and  moral  crises  of  our  time"  (p.278).  Public  debate  both  within  and  without  education 
seems  to  center  on  technical  (as  opposed  to  critical)  issues.  He  suggests  that  this  technical,  instrumental 
educational  discourse  focused  on  technical  issues  should  be  of  concern  to  all  those  who  believe  education 
offers  possibilities  for  hope  toward  overcoming  the  many  current  problems  of  society.  Schools  have  a  role 
to  play,  but  not  the  role. 

This  form  of  educational  discourse  focuses  on  "ameliorating"  the  existing  system,  seeing  it  as 
something  to  be  fixed  or  "Fine-tuned"  but  not  something  to  be  transformed.  Yet  the  urgent  social,  political, 
and  moral  crises  of  our  time  demand  more  than  fixing/fine-tuning.  These  crises  demand  engaging  in  a 
moral  discourse.  Moral  discourse  for  Purpel  means  not  only  "moral  analysis",  but  also  the  task  "must 
include  forging  a  moral  vision-  one  that  can  inform  and  energize  our  political  will  and  educational  strategy" 
(Purpel,  1993,  p.282).  This  "moral  vision"  includes  confronting  "painful  and  anguishing  dimensions  of 
current  educational  practice"  (p.282).  (3)  The  moral  vision  must  include  "the  problematics  and  limitations 
of  the  various  reforms  and  critiques  that  range  from  criticisms  of  teachers  working  conditions  and  ways  to 
basic  curriculum  reform"  (p.282). 

Purpel  (1993)  acknowledges  the  fragmentation  and  isolation  that  indicates  a  crisis  in  community, 
but  rejects  the  notion  of  a  "common  knowledge"  that  binds  people  together.  Rather  we  need  to  "forge  a 
greater  share  of  community  through  a  common  moral  vision"  (p.283). 

At  this  point  I  need  to  present  a  notion  of  democracy  and  community  that  reflects  my  position. 

TOWARD  A  NOTION  OF  DEMOCRATIC  PARTICIPATION 

I  will  identify  features  of  democracy  that  will  outline  my  position: 

1 .  Democracy  must  be  linked  to  a  set  of  values  regarding  such  issues  as  social  justice,  the 
common  good,  equality,  freedom,  community.  These  terms  embody  those  broad  problematic  areas  that 
sometimes  go  by  the  name  of  the  "American  Creed",  or  the  "American  Ideology".  Hence,  to  act 
democratically  means  to  be  guided  by,  to  be  informed  by,  that  set  of  values. 

These  issues  are  problematic.  To  inquire  into  these  issues  requires  a  commitment  to  moral 
discourse. 

2.  These  values  must  be  more  than  abstract  principles,  more  than  intellectual  pursuits,  but  again, 
they  inform  our  work.  Thus,  there  needs  to  be  some  involvement  with  the  "political"  notion  of  "civic 
education". 

This  notion  of  democracy  cannot  be  a-historical.  As  Giroux  (1988)  states,  "Democracy  is  a  'site' 
of  struggle  and  as  social  practice  is  informed  by  competing  ideological  conceptions  of  power,  politics,  and 
community"  (pp.28-29).  To  develop  a  language  of  citizenship  and  democracy  requires  examination  of  the 
"horizontal  ties  between  citizen  and  citizen".  Here  we  are  concerned  with  the  notion  of  difference  wherein 
the  demands,  cultures,  and  social  relations  of  different  groups  must  be  recognized  as  part  of  understanding 
what  it  means  to  be  a  pluralistic  society  (cf  Giroux,  1988,  p. 30).  Thus  "difference  and  identity"  must  be 
central  to  any  debate  regarding  democracy  and  politics.  This  means  that  "theories  of  difference"  are  not  only 
concerned  with  representation  of  identities  (how  the  "Other"  is  represented),  but  also  must  be  concerned  with 
issues  regarding  relations  of  power  (Giroux,  1994,  p.58). 


9 

ERIC 


306 


3 


Identity  here  must  be  seen  as  "...the  effect  of  social  struggles  between  different  communities  over 
issues  of  representation,  the  distribution  of  material  resources,  and  the  practice  of  social  justice"  (Giroux, 
1994,  p.61).  This  understanding  of  identity  exemplifies  what  is  meant  by  democracy  as  a  site  of  struggle. 

What  this  means  for  education  is  that  it  is  not  enough  to  read  the  texts  of  the  "Other".  Wanting  to 
know  the  Other,  according  to  Hazel  Carby  "...  cannot  replace  the  desire  to  challenge  existing  frameworks  of 
segregation".  She  pointedly  asks:  "Have  we,  as  a  society,  successfully  eliminated  the  need  for  achieving 
integration  through  political  agitation  for  civil  rights,  and  opted  instead  for  knowing  each  other  through 
cultural  texts?"  (quoted  in  Giroux,  1994,  p.59). 

This  again  points  to  the  political  notion  of  citizenship.  For  educators  this  means  developing  that  set  of 
values  mentioned  above  that  will  inform  our  work. 

The  challenge  for  educators  is  how  "to  expand  the  basis  for  dialogue  and  community  without 
erasing  a  politics  of  difference"  (Giroux,  1994,  p.59).  Weeks  states 

We  may  not  be  able  :o  find,  indeed  we  should  not  seek,  a  single  way  of  life  that  would 
satisfy  us  all.  That  does  not  mean  that  we  cannot  agree  on  common  political  ends:  the 
construction  of  what  can  best  be  described  as  a  "community  of  communities",  to  achieve 
a  maximum  political  unity  without  denying  difference  (quoted  in  Giroux,  1994,  p.59). 

CONCLUDING  COMMENTS 

I  would  like  to  conclude  with  a  final  comment  on  the  notion  of  democracy  being  a  site  of  struggle. 
Given  the  complexity  of  our  society  today,  the  global  economy,  our  national  economic  outlook,  legislation 
favoring  corporate  interests,  minimum  wage  employment  fueled  by  a  service  economy  and  the  demise  of  the 
middle  class  in  terms  of  living  wages  and  loss  of  industry,  the  rich  getting  richer  and  the  poor  getting 
poorer  and  growing  in  numbers,  t'    backlash  against  multiculturalism  through  a  discourse  of  what  Giroux 
(1994)  calls  "imaginary  unities  of  common  culture  and  national  identity"  (p.57;  also  cf  Schlesinger,  1993), 
etc.,  these  complexities  exemplify  what  is  meant  by  democracy  being  a  site  of  struggle. 

Recognizing  that  text-based  media  can  become  the  basis  for  our  public  discourse  and  can  help 
shape  that  discourse,  we  cannot  confuse  our  creating  and  receiving  texts  through  our  keyboards  with  the 
struggle  for  democracy.  Democracy  is  fought  for  face  to  face.  Democracy  is  fought  for  over  specific  issues. 
The  struggle  for  democracy  is  local,  as  local  and  specific  as  our  workplace. 

Noam  Chomsky,  in  a  recent  interview  (1994),  was  asked  for  tangible  things  for  people  to  do  to  try 
to  change  the  world.  His  response  was 

I  try  to  keep  it  in  the  back  of  my  mind  and  think  about  it,  but  I'm  afraid  that  the  answer 
is  always  the  same.  There  is  only  one  way  to  deal  with  these  things.  Being  alone,  you 
can't  do  anything.  All  you  can  do  is  deplore  the  situation.  But,  if  you  join  with  other 
people,  you  can  make  changes.  Millions  of  things  are  possible,  depending  on  where  you 
want  to  put  your  efforts  (pp.  105- 106). 


As  Chomsky  suggests  here,  and  as  I  have  tried  to  argue  in  this  paper,  it  is  in  community,  in  the  company 
of  others,  where  our  sites  of  struggle  can  make  a  critical  difference. 

ENDNOTES 

1 .  This  has  to  do  with  looking  at  "ideal  types"  which  have  the  essential  characteristics  of  the  issue 
under  discussion  (e.g.  notions  of  freedom,  social  justice,  etc.).  This  form  of  conceptual  analysis  looks  for 
refinement  and  analysis  of  word  use/meaning  in  particular  situations  to  clarify  meaning.  The  end  is  to  be 
more  precise  in  what  a  term  means,  in  order  to  identify  and  correct  ambiguous  or  incorrect  usage  (cf 
Beyer,1988). 
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2.  Here  Beyer  is  referencing  the  over-reliance  on  positivistic  forms  of  inquiry  (empirical  science) 
which  claims  objectivity,  value  neutrality,  and  an  atheoretical  stance.  Within  such  a  position  "knowledge  is 
to  be  found  precisely  by  separating  our  observations  and  analyses  from  that  untrustworthy  social  context 
from  which,  as  Plato  surmised,  only  opinion  can  spring"  (Beyer,  1988,  p.225). 

3.  Purpel  (1994)  is  quite  specific  here  regarding  the  "painful  am'  anguishing  dimensions  of  current 
educational  practice",  and  is  worth  quoting  at  length: 

Teachers  are  caught  up  in  a  system  in  which  individual  achievement,  competition,  success 
and  aggressiveness  are  essential  and  central  elements.  It  is  a  system  in  which  education 
becomes  an  instrument  in  legitimizing  and  defining  hierarchy;  in  which  schools  are  a  site 
where  people  are  sorted,  graded,  classified,  and  labeled,  hence  giving  credence  to  the  tacil 
social  value  that  dignity  is  to  be  earned.  Teachers  are  asked  to  prepare  students  differently- 
some  are  to  be  given  the  encouragement  and  skills  to  be  leaders,  whereas  others  are  taught 
to  endure  their  indignities  quietly  and  proudly.  It  is  a  system  that  helps  sustain  and 
legitimize  a  society  reveling  in  consumerism,  jingoism,  hedonism,  greed,  and  hierarchy" 
(p.282). 
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Abstract 


This  paper  discusses  a  variety  of  development  strategies  and  issues  involved  in  the  development  of 
electronic  performance  support  systems  for  professionals.  The  topics  of  front-end  analysis,  development,  ■ 
and  evaluation  are  explored  in  the  context  of  a  case  study  involving  the  development  of  an  EPSS  to  support 
teachers  in  the  use  of  alternative  assessments.  Strategies  and  concepts  such  as  rapid  prototyping,  formative 
experimentation,  usability,  and  socio-technical  perpectives  are  highlighted. 


Developing  Electronic  Performance  Support  Systems 
for  Professionals 

Professionals  come  to  their  jobs  in  today's  rapidly  changing  workplace  from  a  variety  of  different 
backgrounds  each  with  a  collection  of  different  interests,  skills,  and  expertise.  As  a  result,  educational 
planners  see  the  need  to  provide  multiple  jnd  alternative  educational  experiences,  rather  than  try  to  design 
the  one  best  training  program  (Scribner  &  Sachs,  1991).  A  method  for  meeting  this  challenge  is  to  provide 
people  learning  opportunities  and  experiences  on-the-job  and  at  the  time  of  need  to  bridge  knowledge  gaps 
and  skill  deficiencies  encountered  while  performing  their  jobs  (Geber,  1990;  Gery,  1991;  Law,  1994). 

A  new  tool  integral  to  the  success  of  this  approach  is  what  has  been  coined  by  Gery  (1991)  and 
others  as  an  Electronic  Performance  Support  System  (EPSS).  In  its  most  basic  form,  an  EPSS  is  a 
computer-based  performance  tool  consisting  of  an  informational  database,  an  advisory  system,  models  and 
examples,  and  a  tutorial/learning  base  composed  of  granular-sized  pieces  of  computer-based  training  all 
linked  to  one  another  in  a  relational  database.  The  primary  characteristic  of  this  system  that  allows  the  user 
to  solve  problems  due  to  a  lack  of  knowledge  or  skills  is  the  ability  to  quickly  access  well-structured  pieces 
of  information,  advice,  models,  and  tutorial  assistance  at  the  moment  of  need  as  determined  by  the  user  in 
the  context  of  the  present  performance  problem.  These  systems  range  in  complexity  and  sophistication 
based  on  the  development  resources  available  and  the  nature  of  the  performance  support  context.  To  date, 
most  of  these  systems  have  been  developed  for  use  in  business,  industrial,  and  governmental  and  military 
settings,  however,  this  appears  to  be  changing.  Instructional  technologists  and  other  educators  are  examing 
ways  in  which  these  systems  can  be  developed  to  support  teaching  and  learning  in  educational  settings 
(Brush,  Knapczyk  &  Hubbard,  1993;  Collis  &  Verwijs,  1995;  Goodrum,  Dorsey  &  Schwen,  1993; 
Schwen,  Goodrum  &  Dorsey,  1993). 

The  Learning  and  Performance  Support  Laboratory  at  the  University  of  Georgia  is  currently 
developing  an  EPSS  to  support  school  teachers  in  the  use  of  alternative  assessments.  The  EPSS  is  being 
designed  to  assist  teachers  at  all  levels  of  the  assessment  process  covering  a  range  of  subject  domains  and 
grade  levels.  Among  the  many  activities  involved  in  using  alternative  assessments,  this  EPSS  is  being 
structured  to  support  teachers  in  the  design,  selection,  implementcMon,  evaluation,  or  administration  of  a 
variety  of  performance-based  assessments  coming  under  the  rubric  of  alternative  assessment.  This  paper 
presents  critical  design  issues  that  developers  may  encounter  in  developing  an  EPSS  for  professionals.  The 
Alternative  Assessment  Resource  Center  for  Teacher*  (AARCT)  will  be  presented  as  a  case  study  to 
illuminate  the  issues  discussed.  The  design  issues  are  organized  into  three  main  categories  outlining  the 
development  processes  utilized:  (1)  Front-end  Analysis;  (2)  Development;  and  (3)  Evaluation  and  Research. 

Front-end  Analysis 

Matching  techniques  with  types  of  problems 

Developers  of  EPSSs  are  currently  using  a  combination  of  traditional  instructional  systems 
development  (ISD)  procedures,  software  engineering  principles,  human  factors  principles,  and  a  great  wealth 
of  intuition  based  on  experience  about  what  makes  sense  and  seems  to  work  best  (Gery,  1995;  Witt  & 
Wager,  1994).  Because  different  types  of  professions  require  different  sets  of  skills  (e.g.,  computer 
technicians,  customer  service  representatives,  business  managers,  physicians,  teachers),  developers  of 
EPSSs  are  being  asked  to  support  a  range  of  job-related  skills  from  procedural  tasks  to  complex  problem 
solving.  While  procedural-type  tasks  lend  themselves  nicely    traditional  ISD  front-end  analysis  procedures 
such  as  task  analysis  or  procedural  analysis,  other  tasks  involving  problem  solving  of  ill-defined,  complex 
problems  might  be  better  analyzed,  and  subsequently  better  supported,  using  other  methods  that  examine 
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how  people  in  these  contexts  solve  problems  and  make  decisions  in  which  there  is  usually  no  one  right 
answer. 

Ill-defined,  complex  problems  that  may  require  the  expertise  of  an  experienced  problem-solver  in  a 
particular  domain  (or  work  environment)  are  difficult  to  adequately  train  people  for  using  the  traditional  ISD 
approach.  This  is  because  for  many  ill-structured  or  ill-defined  problems,  heuristics  rather  than  algorithms 
are  required  to  achieve  ends  (Tripp  &  Bichelmeyer,  1990).  A  heuristic  can  be  thought  of  as  a  "rule  of 
thumb"  method  for  solving  problems.  It  is  not  considered  to  be  a  guaranteed  method  for  solving  the 
problem,  but  is  usually  faster  and  more  tractable  than  the  correct  algorithm  (Ashcraft,  1989).  Heuristics  are 
just  one  example  of  the  repertoire  of  reasoning  tools  expert  problem  solvers  have  as  a  result  of  experience 
in  a  particular  domain.  Documenting  and  understanding  the  expert's  reasoning  processes  as  he  or  she 
reviews  past  experiences  to  solve  similar  or  novel  problems  can  be  a  powerful  method  for  learning  how  an 
expert  solves  problem  in  a  performance  context. 

Case-based  Reasoning  for  Analyzing  Complex  Problems 

In  the  absence  of  relevant  task-specific  procedures,  teachers  often  rely  on  instructional  principles 
derived  from  their  experiences  as  teachers,  or  other  teachers  in  the  case  of  teachers  new  to  the  profession,  to 
help  guide  them  in  the  decision-making  process.  This  may  also  involve  adapting  examples  from  support 
materials  such  as  teacher  handbooks,  textbooks,  videotapes,  computer  databases,  or  other  types  of 
curriculum  resource  materials.  Similar  to  other  types  of  professionals  (e.g.,  physicians,  attorneys,  business 
executives,  etc.)  teachers  become  highly  skilled  at  adapting  relevant  examples  (i.e.,  instructional  plans) 
from  one  situation  to  fit  a  new  instructional  problem  that  is  related  in  some  aspect.  These  types  of 
problem  solving  and  decision  making  strategies  indicate,  as  some  researchers  (Kolodner,  1992;  Riesbeck  & 
Schank,  1989)  have  found,  that  experience  plays  an  important  role  in  both  the  development  of  domain- 
specific  problem  solving  skills  as  well  as  the  refinement  of  those  skills  when  novel  problems  are 
encountered.  The  process  of  examining  previous  cases  to  solve  similar  problems  is  referred  to  as  Case- 
based  Reasoning  (CBR). 

Developers  of  the  Alternative  Assessment  Resource  Center  for  Teachers  found  that  a  case-based 
reasoning  approach  was  the  best  method  for  analyzing  the  extensive  and  complex  nature  of  issues  involved 
in  conducting  alternative  assessments.  A  traditional  task  analysis  could  not  easily  absorb  the  vast  array  of 
knowledge,  skills,  and  expertise  that  teachers  brought  to  bear  when  solving  problems  and  making  decisions 
that  usually  involved  many  instructional  variables.  For  example,  portfolio  assessment  requires  the 
evaluation'of  a  broad  spectrum  of  information  for  making  important  decisions  about  a  student's  work.  The 
teacher  must  take  into  account  a  variety  of  considerations  relating  to  each  student's  achievement.  A  task 
analysis  used  to  understand  this  process  may  uncover  some  key  steps  involved  in  the  process,  however, 
much  of  the  context-specific  problem  solving  knowledge  utilized  by  experienced  teachers  may  not  be 
revealed.  As  Laffey  (1995)  notes  in  the  development  of  an  EPSS  for  technicians  at  Apple  Computer,  "the 
(employees)  could  easily  see  how  the  us.»  of  cases  in  (the  EPSS)  mirrored  their  own  informal  strategies  for 
solving  hard  problems.  The  (employees)  saw  value  in  having  access  to  a  rich  and  broad  set  of  real 
experiences  and  in  the  processes  that  keep  the  set  of  experiences  current"  (Laffey,  1995).  Examining  cases 
that  are  set  in  the  context  of  the  work  environment  that  is  to  be  supported  can  provide  developers  of  EPSSs 
a  powerful  method  for  better  understanding  much  of  the  knowledge,  both  explicit  and  implicit,  that 
professionals  utilize  in  performing  their  jobs. 

Case  Study  Approach 


It  became  clear  to  developers  during  the  initial  analysis  phase  of  trying  to  identify  key  tasks  related 
to  conducting  alternative  assessments  that  much  of  the  knowledge  and  skills  that  should  be  supported  were 
actually  evident  in  many  of  the  general  principles  used  by  experienced  teachers.  Therefore,  the  decision  was 
made  to  focus  on  identifying  a  range  of  assessment  examples  that  represented  many  of  the  key  principles 
guiding  teachers  in  the  various  processes.  Once  many  of  these  examples  were  identified,  a  process  for 
documenting  the  cases  using  a  multimedia  format  was  determined.  The  use  of  multimedia  case  studies  as  a 
framework  to  both  gather  data  and  structure  support  information  was  decided  to  be  the  primary  development 
strategy  for  collecting  and  organizing  other  support  materials  within  the  system. 


Data  Collection 


A  systematic  data  collection  tool  was  developed  for  focusing  efforts  on  how  to  best  document  cases 
when  visiting  the  schools.  Because  time  is  alv/ays  at  a  high  premium  for  teachers,  decisions  such  as  what 
kinds  of  media  elements  to  be  gathered  were  made  prior  to  arriving  at  the  school  by  using  the  data  collection 
worksheet.  Developers  conducted  pre-interviews  with  teachers  to  get  an  idea  of  the  types  of  elements  that 
should  be  highlighted  in  cases,  as  well  as  to  prepare  teachers  for  videotaped  interviews  about  their  role  in 
the  assessments.  This  methodology  for  collecting  a  source  of  rich  data  for  multimedia  case  studies  proved 
to  be  helpful  in  maximizing  limited  time  constraints. 

Development  Issues  and  Strategies 

Rapid  Prototyping  Methodology 

The  developers  of  AARCT  took  the  position  from  the  beginning  that  developing  an  EPSS  for 
teachers  had  to  be  a  participatory  design  process  involving  the  teachers  if  the  product  was  ever  going  to  be 
anything  that  teachers  would  really  use.  Also,  to  achieve  a  high  level  of  usability  there  would  also  have  to 
be  a  development  process  that  afforded  considerable  amounts  of  iteration  involving  conceptualizing, 
developing,  testing,  and  revising.  This  iterative  design  process  which  is  common  in  software  engineering 
is  referred  to  as  prototyping.  When  the  process  of  constructing  prototypes  is  accelerated,  so  that  the  time 
from  beginning  a  prototype  to  evaluating  user  interaction  with  it  is  short  enough  to  allow  time  for 
substantial  changes  to  the  product,  then  this  process  is  usually  termed  rapid  prototyping  (Hix  &  Hartson, 
1993). 

A  rapid  prototyping  methodology  was  chosen  to  afford  developers  the  creativity  and  flexibility  to 
"get  it  right"  referring  to  desired  usability  standards.  While  the  idea  of  creating  the  "perfect"  interface  may 
be  unrealistic,  the  use  of  software  engineering  principles  such  as  prototyping  and  usability  testing  are  ways 
to  increase  the  likelihood  that  users  will  be  able  to  use  the  system  successfully.  This  approach  appears  to 
be  well  supported  in  the  EPSS  literature  as  mostly  anecdotal  evidence.  As  Gery  (1995)  points  out,  "few 
(designers)  are  guided  by  a  set  of  integrated  and  fully  articulated  design  principles.  Many  innovations  are  the 
result  of  individual  or  team  creativity  and  iterative  design  employing  rapid  prototyping  coupled  with 
ongoing  usability  and  performance  testing". 

The  first  prototype  proved  to  be  a  valuable  lesson  in  usability  testing.  Developers  constructed  a 
working  prototype  using  an  authoring  system  that  allowed  them  to  test  information/knowledge  organization 
■jnd  navigational  strategies  (See  Figure  I).  A  school  building  metaphor  was  used  as  an  interface  design. 
Also  accompanying  this  were  a  panel  of  menu  buttons  that  forced  the  user  to  delimit  choices  based  on 
assessment  type,  grade  level,  and  content/subject  area.  A  focus  group  of  fifteen  school  teachers  viewed  a 
demonstration  of  the  system  that  had  a  few  example  pieces  hard  linked  to  one  of  the  cases.  After  going 
through  the  initial  demonstration  sequence,  the  developers  were  soon  made  aware  by  a  majority  of  the 
teachers  that  the  delimiting  strategy  of  narrowing  down  choices  based  on  pre-defined  categories  was  not  only 
counter-intuitive  but  represented  antithetical  ways  of  thinking  about  instruction.  Most  of  these  teachers 
taught  interdisciplinary  units  that  combined  topics  from  a  wide  variety  of  content  domains,  therefore,  the 
navigation  structure  did  not  support  the  way  they  thought  about  instruction  (or  assessment).  In  addition, 
some  of  the  representative  teachers  taught  more  than  one  grade  level  and  did  not  like  the  "arbitrary  selection" 
of  grade  levels.  This  could  be  thought  of  as  a  discrepancy  between  the  developers'  mental  models  of  this 
domain  and  the  teachers'  mental  models.  A  mental  model,  simply  put,  is  the  learner's  naturally  evolving 
model,  or  internal  representation,  which  summarizes  his  or  her  own  knowledge  about  a  process  or  thing, 
often  taking  the  form  of  an  informal  theory  (Norman,  1983).  Mental  models  are  representations  that  are 
active  while  solving  a  particular  problem  and  that  provide  the  workspace  for  inference  and  mental  operations 
(Halford,  1993).  The  teachers  asked  for  an  interface  that  reflected  their  mental  models  ~  an  interdisciplinary 
approach  to  content  domains,  grade  levels,  and  assessment  strategies.  Their  mental  models  of  how  the 
interface  should  look  reflected  how  they  think  about  their  jobs  and  the  tasks  involved.  This  information 
was  lacking  for  the  developers  and  became  evident  in  their  interface  design. 

In  summary,  it  is  useful  to  first  interview  intended  users  to  get  an  idea  at  some  level  about  how 
they  think  about  a  task  or  performance  context.  Then,  start  with  an  idea  and  test  its  conceptual  validity 
with  a  sample  of  intended  users  before  launching  into  any  high-level  screen  design.  By-passing  this 
sequence  may  only  prove  that  you  have  a  different  mental  model  about  a  type  of  work  environment  and  its 
associative  tasks  than  someone  who  actually  works  in  that  particular  performance  context  (See  Carroll  & 
Olson,  1984). 
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Evaluation  Issues  and  Strategies 

Formative  Experimentation 

A  strategy  that  works  well  with  the  goals  of  rapid  prototyping  to  include  the  intended  users  in  the 
design  process  is  called  formative  experimentation.  Newman  (1990)  describes  formative  experiments  as  a 
type  of  research/design  method  that  "sets  a  pedagogical  goal  and  finds  out  what  it  takes  in  terms  of 
materials,  organization,  or  changes  in  the  technology  to  reach  the  goal".  In  this  way  design,  formative 
evaluation,  and  research  are  all  tied  together  and  directed  toward  reaching  a  goal.  In  this  case,  developers  of 
the  AARCT  are  examining  ways  in  which  to  best  support  teachers  from  a  variety  of  backgrounds  and  levels 
of  expertise  in  the  use  of  alternative  assessments.  It  is  believed  to  really  achieve  this  that  developers  will 
need  to  work  closely  with  teachers  using  prototypes  of  the  EPSS  to  see  how  they  are  using  it,  if  at  all. 

As  others  have  suggested  (Goodrum  et  al.,  1993),  by  examining  the  socio-technical  aspects  of 
technology  in  the  workplace,  it  becomes  important  to  recognize  that  technology  is  only  one  critical,  highly 
interactive,  and  interdependent  component  of  the  whole  organization  or  setting.  When  considering  the  use 
of  an  EPSS  to  support  performance  in  a  work  environment  it  is  necessary  to  consider  all  of  the  elements 
and  methods  that  make  up  the  way  in  which  work  is  conducted.  Developers  of  EPSS  need  to  examine  the 
social,  cultural,  and  organizational  aspects  that  affect  the  way  in  which  work  is  performed  in  the  work 
environment.  By  using  methods  like  formative  experimentation,  developer  can  "test"  and  "tweak"  systems 
to  take  into  account  many  of  these  factors  influencing  work. 

Tiie  developers  of  the  AARCT  are  interested  in  seeing  not  only  how  teachers  are  using  the  system 
to  support  them  in  the  use  of  alternative  assessments,  but  they  are  also  interested  in  seeing  how  teachers 
will  restructure  their  work  as  a  result  of  having  this  system  available.  Will  they  make  it  part  of  their 
normal  planning  time  to  utilize  the  system?  Will  they  use  the  system  to  help  manage  the  tasks  involved  in 
alternative  assessments?  Will  they  share  information  learned  about  the  system  with  other  teachers?  Will 
they  find  it  desirable  to  update  the  system  periodically  to  include  new  cases?  These  are  just  some  of  the 
questions  that  the  developers  of  the  AARCT  will  examine  as  the  system  is  tested  in  a  few  of  the  schools 
participating  in  the  project.  Many  other  questions  will  be  posed  throughout  the  formative  experimentation 
process  to  see  how  the  goal  of  supporting  teachers  in  these  tasks  can  best  be  achieved  with  the  EPSS. 

Conclusion 

Developers  of  EPSSs  should  examine  a  variety  of  development  strategies  in  an  attempt  to  find  out 
what  best  works  for  their  situation.  The  developers  of  the  AARCT  are  utilizing  principles  found  in 
instructional  systems,  software  engineering,  performance  technology,  and  formative  experimentation  to 
develop  an  effective  EPSS  for  teachers.  There  are,  of  course,  many  other  strategies  that  could  prove  to  be 
valuable  as  well  in  this  and  in  similar  endeavors.  What  is  important  to  keep  in  mind,  however,  is  that  by 
making  usability  a  goal  and  focusing  on  the  social,  cultural  and  organizational  factors  that  influence  how 
work  is  performed  by  professionals  in  a  specific  work  context,  developers  increase  the  likelihood  of 
developing  effective  EPSSs  that  support  the  people  and  the  organization  in  meeting  their  goals. 
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Abstract 


This  article  lists  and  briefly  defines  fourteen  values  that  summarize  the  constructive  framework.  It  is 
suggested  that  the  values  provide  a  basis  for  evaluating  instructional  strategies  aimed  at  fostering  higher- 
order  thinking  skills  and  positive  disposition  toward  learning.  It  is  further  suggested  that  the  values  may 
lead  to  a  set  of  guidelines  for  developing  a  wide  variety  of  learning  environments. 


Constructivist  Values  and  Emerging  Technologies: 
Transforming  Classrooms  into  Learning  Environments 

As  emerging  technologies  offer  the  potential  for  designing  highly  engaging,  student-centered  learning 
environment  and  as  the  theory  base  for  situated  learning  matures,  a  number  of  educational  approaches  have 
evolved  that  foster  higher-order  thinking  skills  and  positive  disposition  toward  eanunfr  These  include 
cognitive  apprenticeship  (Brown,  Collins,  &  Duguid,  1989),  anchored  instruction  (CTGV    990),  The 
Community  of  Learners  (Brown  &  Campione,  1990),  and  Computer-Supported  Intentional  Learning 
Environments  (Bereiter  &  Scardamalia,  1992),  to  name  a  few.  The  educational  problem  that  this  paper 
addresses  is  how  to  summarize  the  constructivist  framework  reflected  in  these  and  related  approaches,  in  a 
way  that  provides  educators  with  manageable  guidelines  for  transforming  classrooms  into  computer- 

ST"urS^ 

thinking  defined  as  the  process  of  figuring  out  what  to  believe  or  do  about  a  problem  or  which  no  sing  e 
defm  five  answer  exists  and  as  the  process  of  evaluating  the  quality  of  text  and  other  intellectual  product 
inSng  one's  own  (Brown  &  Campione,  1990),  (b)  generative  thinking,  defined  as  the  active  process  of 
using  existing  knowledge  to  make  sense  of  new  information  necessary  for  meaningful  learning  defined  as 
EffiS  declarative  knowledge  to  use-oriented  knowledge  (Wittrock,  1991 ;  Bradford  &  Vye 
1989)  (c  problem-solving  processes,  defined  as  the  application  of  current  knowledge  to  delineate  and  think 
out  a  sdutkm  to  a  novel  problem  (Gagne,  1984),  and  (d)  the  use  of  cognitive  and  metacogmhve  strangle 
for  achieving  learning  goals  and  exercising  executive  control  of  concentration,  comprehension,  and  affect  ( 
Weinstein  &  Mayer,  1986;  Corno,  1986,  Shuell,  1986). 

Positive  disposition  toward  learning  refers  to  individual  differences  in  personal  agency  and  learning 
orientation  thatinfluence  students'  perceptions  of  the  learning  situation,  the  quality  of  their  cognitive 
engagement  and  whether  their  learning  activities  lead  to  meaningful  learning.  Personal  agency  is  the 

en8de8ncTfor  Splc  "to  take  responsibility  for  their  actions  and  ascribe  success  and  failure  to  he  goals  *ey 
choose  the  resources  they  mobilize,  and  the  effort  they  expend"  (Pans  and  Byrnes  1989,  p.  177)  Learning 
orStaiion  includes  an  individual's  beliefs  about  the  nature  of  knowledge  and  how  it  is  acquired,  the  type  of 

gXu  sued  by  the  individual  on  cognitive  tasks,  and  the  individual's  perceptions  about  task  requirements. 

ft  encompasses  expectations,  feelings,  intuitions,  attitudes,  values,  interests,  intentions,  significant 

relationships,  and  commitments  (Vermunt,  1989). 

Ser  (1988)  has  observed  that  the  values  we  hold  as  educators  influence  our  students  in  ways  that 

crucially  affect  their  development.  He  has  proposed  that  "the  methods  we  espouse,  the  way  we  define 

knowledge  Se  work  we  regard  as  respectable,  reflect  our  conceptions  of  virtue,  and  the  courses  we  teach  n 

S  Cd  lp  students  ^  such  virtueS"  (PJ9)-  TaylW  and  SW3rt1  ?8)  T  Xlt'n 
educafional  technology  is  not  as  value-neutral  a  method  of  conveying  instructional  information  as  has  been 

common^ ,  believed.  Rather,  educational  technology  has  generally  supported  a  set  of  values  identified  by 

Heinich  (1984)  as  replicability,  reliability,  communication,  and  control  (see  figure  1). 
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Figure  1-  Contemporary  values  of  educational  technology  as  identified  by  Heinich,  (1984). 
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In  the  face  of  what  Doll  (1989)  has  described  as  an  emerging  post-modern  agenda  for  curriculum,  an 
alternative  (but  not  necessarily  mutually  exclusive)  set  of  values  has  appeared  (see  figure  2).  These  values, 
as  defined  in  the  following  section,  come  primarily  from  a  review  of  literature  in  the  areas  of  constructivist 
principles  of  teaching  and  learning,  contemporary  learning  theories,  achievement  motivation  (especially 
goal  theory),  collaborative  learning  environments,  and  the  role  of  computers  and  related  technology  in 
education. 
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Figure  2.  COMP:  Constructivist  values  for  developing  higher-order  thinking  skills  and  positive 
disposition  toward  learning. 


© 


Ownership 

Appropriation 


Personal 
Autonomy 


Community 


Self- 
reg  illation 


Reflectivity 


Personal 
Relevance 


©Collaboration 
Mutual  Inquiry 


Fi  rat  do 
no  harm. 


Meaning 

Generativity 


Pluralism 


.Pe 


Multiple 
rspectives 


Transformation 

Active 
Engagement 


Complexity 


Practice 

Authenticity 

® 


It  should  be  noted  that  many  of  the  values  are  consonant  with  elements  of  both  experiential  education,  as 
advocated  by  Dewey  (1972)  and  the  progressive  educators  of  the  1920s  and  30s,  and  humanistic  learning 
theory,  as  championed  by  Rogers  ( 1 963 )  and  Maslow  ( 1 968). 

1 .  Active  engagement.  A  quality  of  mindfulness  in  learning  situations  that  supports  intentional 
learning  by  influencing  the  learning  strategies  the  individual  employs.  Intentional  learning  refers  to 
cognitive  processes  that  involve  purposeful  processing  of  information  to  achieve  a  learning  goal  actively 
desired  by  the  learner  (Bereiter  &  Scardamalia,  1989). 

2.  Authenticity.  A  characteristic  of  learning  environments  that  are  designed  to  develop  understanding 
through  application  and  manipulation  of  knowledge  within  the  context  of  the  ordinary  practices  of  the  target 
culture  (Brown  et  al.,  1989).  Authenticity  is  concerned  not  only  with  the  similarity  of  the  learning 
situation  to  the  transfer  environment,  but  also  with  the  value  learners  attribute  to  the  learning  task  and 
whether  the  learner  practices  what  is  essential  for  the  transfer  situation. 

3.  Collaboration.  Various  noncompetitive  social  interactions  that  affect  cognitive  development,  viewed 
on  a  two  dimensional  scale  ranging  from  high  to  low  levels  of  united  effort  and  continuous  sharing  and 
from  symmetric  to  highly  asymmetric  relative  expertise  of  participants  (Granott,  1993).  When  mutual 
inquiry  is  the  general  framework  for  learning,  traditional  roles  of  teacher  and  student  are  largely  replaced  by  a 
collaborative  partnership  in  a  many-sided  inquiry  into  meaning  (Doll,  1989). 

4.  Community.  The  emphasis  in  various  conceptions  of  affiliated  work  that  focus  on  jointly 
constructed  public  knowledge  as  opposed  to  individual  knowledge  (Brown  &  Campione,  1990).  This 


orientation  represents  a  fundamental  change  in  education,  where  the  goal  is  to  transform  classrooms  into 
knowledge-building  communities  that  allow  students  to  contribute  to  each  other's  learning  through  the 
social  construction  of  communal  knowledge  (Bereiter  &  Scardamalia,  1992). 

5.  Complexity.  The  view  that  reality  is  web-like,  with  multiple  interacting  forces  (Doll,  1989)  and 
that  many  areas  of  human  endeavor,  particularly  those  involving  interaction  between  people,  are  ill- 
structured  and  characterized  by  a  high  degree  of  concept  and  case  complexity  and  across-case  irregularity 
(Spiro,  Feltovich,  Jacobson,  &  Coulson,  1991).  • 

6.  Generativity.  The  active  process  of  using  existing  knowledge  to  make  sense  of  new  information 
necessary  for  meaningful  learning.  In  generative  teaching  for  understanding,  the  teacher's  role  is  to  help 
students  build  connections  between  their  knowledge,  beliefs,  and  experience  on  the  one  hand,  and  school 
subject  matter  on  the  other.  Within  this  model,  comprehension  depends  on  the  activities  of  the  learner 
during  instruction.  The  learner  creates  relations  between  prior  knowledge  and  current  experience  by 
applying  learning  strategies  such  as  elaborating  with  analogies,  making  inferences,  paraphrasing,  and 
summarizing  (Wittrock,  1991). 

7.  Multiple  perspectives.  Experiencing  the  same  material  in  a  variety  of  different  ways  and  for  different 
purposes.  Students  develop  flexible  knowledge  by  experiencing  multiple  cases  from  multiple  perspectives 
(Spiro  etal.,  1991). 

8.  Ownership.  Experiencing  the  value  of  new  understanding  and  the  benefits  of  personal  agency  in 
developing  commitment  to  meaningful  learning  goals  and  the  ability  to  achieve  these  goals  through  self- 
regulation  of  the  learning  process. 

9.  Personal  autonomy.  The  quality  or  condition  of  feeling  in  charge  of  the  learning  process  as  opposed 
to  expecting  others  to  direct  it.  Support  of  autonomy  (as  opposed  to  control)  by  providing  choice, 
minimizing  performance  pressure,  and  encouraging  initiation  tends  to  support  intrinsic  motivation, 
meaningful  learning,  self-esteem  and  a  variety  of  other  factors  relevant  to  the  initiation  and  regulation  of 
intentional  learning  (Deci  &  Ryan,  1987). 

10.  Personal  relevance.  Experiencing  changes  in  perceptions,  understandings,  beliefs,  feelings,  and 
capabilities  as  a  function  of  new  information  (as  opposed  to  being  told  about  its  relevance).  An  individual's 
orientation  toward  learning  is  qualitatively  different  when  learning  is  embedded  in  the  context  of  achieving 
personally  relevant  goals  that  go  beyond  working  for  a  grade  or  some  distant,  future  goal  (Berliner,  1992; 
Bransford,  Franks,  Vye,  &  Sherwood,  1989). 

1 1 .  Pluralism.  The  belief  that  no  single  view  of  reality  can  explain  all  the  phenomena  of  life,  and, 
therefore,  reality  is,  ultimately,  the  product  of  intersubjective  agreement.  From  this  perspective,  each 
individual's  perceptions  have  value  and  represent  a  basis  for  mutual  inquiry,  rather  than  an  obstacle  to  be 
maneuvered  around  (Doll,  1989). 

12.  Reflectivity.  Largely  synonymous  with  metacognitive  awareness,  defined  as  the  ability  to  plan, 
monitor,  and  modify  cognitive  processes  for  mental  management  or  executive  control  of  concentration, 
comprehension,  anr1  affect  (Corno,  1986;  Osman  &  Hannafin,  1992;  Shuell,  1986).  Students  become 
increasingly  responsible  for  learning  through  strategic  exploration  of  errors  and  thoughtful  consideration  of 
their  own  and  others'  knowledge-construction  activities. 

13.  Self-regulation.  A  form  of  cognitive  engagement  involving  metacognitive,  motivational,  and 
behavioral  dimensions  for  assuming  personal  responsibility  for  learning  (Corno  &  Mandinach  1983; 
Zimmerman,  1989).  Constructivists  hold  that  the  ultimate  goal  of  education  is  to  help  students  become 
masters  of  their  own  learning  by  supporting  the  progressive  acquisition  of  the  capability  to  self-regulate 
learning. 

14.  Transformation.  A  view  of  change  that  sees  meaningful  learning  as  the  result  of  internal 
reorganization  triggered  by  the  learner  or  by  the  learner's  reaction  to  external  forces  rather  than  a  process  of 
accumulating  information  in  memory  (Doll,  1989). 

The  development  of  guidelines  based  on  this  set  of  values  represents  a  strategy  for  summarizing  the 
constructivist  framework  in  a  way  that  can  lead  to  a  wide  variety  of  learning  environments  and  further 
provides  a  basis  for  evaluating  instructional  strategies  (Savery  &  Duffy,  in  press).  From  another 
perspective,  these  values  are  concerned  with  epistemology,  which  as  Bateson  (1972)  has  suggested, 
ultimately  defines  the  kind  of  world  we  live  in. 
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The  purpose  of  this  study  was  to  investigate  the  status  of  computer  usage  within  Korean  and 
United  States  schools,  and  then  use  the  inte.net  to  establish  cross  cultural  communications  between 
schools.  We  hope  that  through  this  international  communication  using  the  internet,  students  at  the 
elementary,  junior  high,  and  high  school  levels  will  gain  deeper  knowledge  about  other  cultures  and  develop 
higher  order  thinking  skills  as  they  engage  in  collaborative  projects.  The  Internet  allows  people  to  use 
computers  to  communicate  by  using  the  existing  telephone  lines;  messages  can  be  sent  and  received 
between  various  parts  of  the  world,  wherever  the  telephone  lines  travel  and  Internet  access  has  been 
obtained. 

After  numerous  planning  meetings  and  site  visits  to  schools,  this  project  is  in  the  preliminary 
stages  of  Internet  communications  between  students.  Eventually,  we  hope  students  can  travel  to  visit  their 
counterpart  schools  and  thus  have  first  hand  experience  within  another  culture.  This  cross  cultural  use  of 
the  internet  has  great  potential  to  enhance  the  richness  of  student  knowledge  and  thinking  by  allowing 
multiple  ideas  to  flow  from  different  perspectives,  while  developing  multicultural  sharing  and  acceptance 
from  a  young  age,  thus  circumventing  some  of  the  preconceived  prejudices  found  in  the  world  today.  Such 
understanding  and  tolerance  can  most  certainly  influence  future  political,  business,  and  educational 
decisions,  and  policies.  This  goal  goes  far  beyond  the  current  attempts  at  understanding  world  news  and 
cultural  events  from  an  isolated  standpoint. 

The  sharing  that  can  take  place  between  students  in  the  different  countries  has  the  potential  to 
develop  collaboration  at  an  entirely  different  level  than  has  been  possible  in  the  past.  Although  there 
potentially  have  been  other  projects  in  the  past  that  might  have  encouraged  such  inter-cultural 
communication,  they  have  been  largely  limited  to  print  or  video  based  efforts,  and  have  had  delays  in  both 
the  type  and  speed  at  which  communication  could  take  place.  The  internet  allows  the  most  immediately 
interactive  method  of  collaboration  at  a  distance,  that  transcends  past  efforts  by  allowing  students  to  take  on 
joint  efforts  on  projects.  For  example,  students  might  engage  in  interactive  school  policy  discussions,  or 
perhaps  design  together  a  project  that  in  the  end  reflects  characteristics  of  different  styles  of  the  different 
countries  involved.  Such  a  project  would  reflect  a  truly  multicultural  result  for  no  matter  what  the  topic, 
students  could  bring  their  own  perspective  to  the  task  and  thus  the  input  would  represent  multicultural 
thinking.  The  potential  impact  of  different  viewpoints,  artistic  expression,  value  systems,  and  so  on  offered 
through  such  a  project  could  result  in  thinking  that  goes  beyond  that  possible  within  a  more  limited 
setting.  Further,  you  can  imagine  the  difference  between  studying  about  Eastern  or  Western  cultures  by 
using  traditional  methods  versus  having  current  cultural  information  provided  by  one's  peer  age  group  that 
lives  within  the  other  culture.  For  example,  the  study  of  music  would  no  longer  be  limited  to  the  music 
classroom  or  traditional  musical  forms  associated  with  any  particular  culture,  but  could  include  the  favorite 
popular  music  and  artists  as  recognized  by  each  country's  current  youth;  a  new  car  design  could  contain 
some  of  the  features  currently  found  on  cars  within  each  country;  discussions  on  social  policy  could  bring 
explanations  of  the  background  reasons  for  particular  customs  or  viewpoints;  and  anthropological 
information  such  as  viewpoints  on  aging  and  family  units  could  bring  a  deeper  understanding  of  other 
cultures. 

Since  the  1970s,  schools  in  the  United  States  have  struggled  to  find  meaningful  ways  to 
implement  computers  in  order  to  respond  to  a  societal  need  to  develop  computer  literacy  along  with  skill  in 
using  the  newly  available  information  technologies.  These  technologies  are  changing  rapidly  and  schools 
are  struggling  to  keep  abreast  of  those  changes.  Thus  many  schools  are  interested  in  expanding  their 
activities  to  include  more  challenging  and  educationally  meaningful  experiences  for  students  that  will 
translate  directly  into  building  higher  levels  of  thinking  and  better  skills  at  using  the  newly  available  and 
powerful  information  technologies.  Telecommunications  has  been  an  important  method  of  offering  distance 
education  for  adult  and  high  school  audiences  over  the  past  several  years,  but  very  little  has  been  done  using 
the  internet  for  these  audiences  and  very  little  has  been  attempted  to  create  educational  opportunities  for 
elementary  and  middle  school  students  using  telecommunications.  One  goal  of  this  project  was  to  establish 
an  international  communications  link  between  school  students  at  the  elementary,  junior  high,  and  high 
school  levels,  thus  empowering  students  to  go  beyond  the  traditional  curriculum  and  gain  education  in  a 
different  way. 

The  important  issues  surrounding  educational  computing  have  changed  over  the  past  decade  as  the 
technology  and  access  to  ii  have  improved.  When  computers  were  first  placed  in  U.S.  schools,  the 
questions  focused  on  how  to  get  more  hardware  and  software.  Then,  as  more  and  better  equipment  became 
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available,  the  focus  shifted  attention  to  the  best  wa  •  to  implement  computers  for  educationally  rich 
experiences.  Databases  have  made  huge  caches  of  information  available  to  students  and  the  teaching  process 
itself  has  shifted  to  focus  on  building  higher  order  thinking  and  research  skills,  changing  the  very  nature  of 
the  teacher's  job. 

As  the  schools  struggle  to  find  meaningful  uses  for  computers,  the  technology  often  drives  the 
curriculum  effort  (Knupfer,  1993).  It  is  apparent  that  the  curriculum  needs  should  lead  the  technological 
efforts,  but  the  technology  offers  unique  opportunities  that  go  beyond  the  original  curriculum  plans.  Many 
countries  wrestle  with  these  issues. 

In  the  mid  1980s  Korea  began  to  implement  computers  to  promote  information  technology  skills 
in  schools.  In  Korea,  the  effort  was  led  by  social  forces,  much  the  same  as  in  the  U.S.  But  what  stage  is 
Korea  at  now?  How  far  have  U.S.  schools  progressed  with  information  technologies?  Which  country  is 
leading  the  telecommunications  efforts?  Both  countries  have  produced  leaders  in  researching  and 
implementing  educational  computing,  and  scores  of  Korean  teachers  have  pursued  graduate  studies  in  U.S. 
universities  to  gain  knowledge  about  educational  communications  and  technology.  How  do  the  countries 
compare  in  their  current  activities  and  efforts  within  the  schools?  What  similarities  and  differences  help  or 
hinder  their  efforts?  What  can  we  learn  from  each  other,  now  that  we  can  communicate  via  the  internet 
communications  system?  What  new  experiences  can  we  offer  our  students  for  learning  jbout  another 
country  and  culture,  and  applying  higher  order  thinking  to  meaningful  inter-cultural  exchange? 

This  paper  attempts  to  answer  some  of  those  questions  as  it  reports  about  some  current  efforts  of 
schools  in  Korea  and  the  United  States.  Specifically,  the  remainder  of  this  paper  will  describe  the  Asian 
Link:  Telecommunications  for  the  21st  Century  project  and  some  efforts  within  the  state  of  Kansas  to  get 
schools  online. 


Asian  Link  :  Telecommunications  for  the  21st  Century 

The  Asian  Link:  Telecommunications  for  the  21st  Century  project  was  initiated  by  the  Australian 
Asian  Education  Foundation  which  was  founded  to  introduce  Australia  to  Asian  countries  and  Asian 
countries  to  Australia  through  education.  They  wanted  to  conduct  this  mission  through  interesting  and 
exciting  methods,  and  internet  telecommunications  received  the  attention  due  to  its  convenience  and 
economic  outlook.  While  the  use  of  telecommunications,  particularly  international  telecommunications 
such  as  that  offered  by  the  internet,  was  actively  applied  in  tertiary  education,  primary  and  secondary 
education  also  can  greatly  benefit  from  it.  Such  experience  will  be  the  basis  for  creating  a  model  which  at 
the  beginning  stage  will  facilitate  closer  ties  between  Korean  schools  and  other  schools  in  Asia,  Australia, 
and  the  United  States  of  America. 

The  purpose  of  this  international  project  is  to  investigate  how  to  apply  telecommunication 
technology  in  K-12  education.  Instructional  activities  through  the  telecommunication  technology  can 
encourage  cooperative  learning  environment  because  activities  in  this  project  are  based  on  the  students' 
choice  and  consensus  among  partners  from  the  school  of  foreign  countries.  It  is  all  based  on  such 
democratic  processes. 

Participating  Countries  and  Supporting  Organizations 

•  Korea:  Korea  Educational  Development  Institute 

•  USA:  Copen  Family  Fund,  Korea  Society 

I#EARN  (International  Education  and  Research  Network) 

•  Australia:  Asia  Education  Foundation  (AEF),  Australia  Korea  Foundation 

I#EARN  Australia 

•  Japan:  Association  for  Promotion  of  International  Co-operation  in  Tokyo 

•  China:  Chinese  Institute  of  Educational  Research  in  Beading 
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What  Schools  Need  to  Prepare 

•  Staff  and  student  ideas 

•  Staff  that  includes  a  group  of  teachers,  especially  an  English  teacher  and  computer  teacher  or  a  teacher 

proficient  in  the  use  of  computers, 

•  Other  teachers  of  any  subject  with  passion  on  this  approach 

•  A  computer  and  telecommunication  software 

•  A  modem  and  phone  line 

•  Teleconferencing  facilities  (Lumba  video  phone  -  slow  scan  black  &  white  phone,  not  essential) 

•  Subscription  to  I#EARN  (International  Education  And  Resource  Network) 

The  computer  and  phone  lines  are  encouraged  tc  be  situated  in  classrooms.  That  is  essential  if  the 
computer  is  to  be  viewed  as  an  integral  part  of  what  is  done  on  a  daily  in  classes.  I#EARN  is  designed  to 
create  an  educational  framework  specifically  for  classroom  teachers  to  utilize  the  vast  resources  available 
through  these  global  networks.  I#EARN  was  created  by  the  Copen  Family  Fund  in  1990,  working 
closely  with  affiliate  telecommunications  networks  of  the  Association  for  Progressive  Communications 
(APC)  system  and  a  number  of  educational  organizations.  Each  participating  school  has  two  internet  IDs 
assigned  by  KEDI  and  one  APC  user  identification  (ID)  supported  by  the  Copen  Family  Fund.  (The  list  of 
IDs  for  each  school  is  listed  in  the  appendix.) 

Instructional  Activities:     Research  Content 

What  is  important  is  the  individual  subject  area,  not  the  computer  or  the  technology. 
Telecommunication  should  be  accepted  as  an  educational  media  to  empower  students  and  teachers,  thus 
hopefully  elevate  the  quality  of  education.  Given  the  enormity  of  global  telecommunications  networks,  it 
would  be  a  daunting  task  (but  not  impossible)  for  an  individual  teacher  to  develop  the  familiarity  and 
contacts  to  create  meaningful  and  relevant  networks  to  both  their  subject  areas  and  student  interests. 

Instructional  activities  are  not  fixed;  they  are  up  to  the  participants.  To  promote  cultural 
awareness,  to  integrate  telecommunications  into  foreign  language  study  at  an  early  age,  and  to  support  inter- 
curricular  activities,  cooperative  project  based  programs  are  among  encouraged. 

How  Students  And  Teachers  Work  Through  APC  Net 

The  participating  schools  are  paired  and  encouraged  to  correspond  to  those  one-to-one  partnership 
schools.  However,  they  are  also  open  to  communicate  with  other  I#EARN  schools  over  the  world 
(approximately  more  than  400  schools  from  23  countries). 

Participating  students  and  teachers  use  conference  rooms  on  APC  network  such  as  AEF.ideas, 
AEF.teachers,  and  AEF.students  in  order  to  exchange  project  ideas.  Ideas  are  tossed  to  those  conferences  and 
anyone  who  are  interested  in  that  idea  can  join  the  project. 

This  communication  is  to  establish  a  long  term  relationship  and  hopefully  result  in  exchanges  and 
other  forms  of  collaborative  learning.  As  a  result  of  this  project,  Ahyuri  Middle  School  (one  of  the 
participating  schools  in  Korea)  sent  31  students  and  4  teachers  to  their  partner  school,  Castle  Hill  High 
School,  in  Sydney,  Australia.  For  the  return  visit,  students  and  teachers  of  Castle  Hill  High  School  will 
visit  Ahyun  Middle  School  next  year  and  they  already  started  their  preparation  such  as  to  offer  classes  for 
Korean  language  which  is  one  of  their  four  major  foreign  languages  reguiated  by  the  Australian  government 
curriculum. 

The  Difficulties  Of  Participating  Schools  Experience 

There  are  numerous  obstacles  to  implementing  the  internet  technology  in  education  in  Korea 
including  the  following: 

1.  Shortage  of  a  phone  line  for  networking 

Generally  public  schools  have  two  three  phone  lines  tor  the  whole  school.  While  the  phone  line 
is  so  scare  resource,  it  is  difficult  to  keep  a  line  for  long  for  networking.  In  most  of  the  schools,  the 
hardware  facilities  for  networking  such  as  a  computer  and  modem  for  networking  is  not  a  problem  but 
installation  of  a  new  designated  phone  line  for  the  networking  creates  problems  like  fare  distribution  of 
school  resources  and  maintenance  fee  monthly. 
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2.  Language  barrier 

English  is  the  second  language  and  in  fact  to  Koreans  learning  English  is  almost  the  most  difficult 
language  to  learn.  Those  students  in  middle  school  who  just  started  to  learn  Englisn  can  experience 
difficulties  in  expressing  themselves  in  the  foreign  language.  In  Korea,  English  is  taught  from  the  middle 
school,  thus  elementary  students  do  not  even  start  to  learn  English.  Teachers  have  the  same  language 
problem.  Although  English  teachers  are  encouraged  in  join  this  project,  their  support  is  limited. 

3.  Speed  of  networking 

In  local  networking,  speed  is  not  such  a  problem.  But  when  it  is  the  international  networking, 
the  speed  becomes  a  problem.  Particularly  for  multimedia  data  networking  (WWW  networking),  it  is  so 
slow  that  schools  can  not  afford  the  phone  bills.  It  is  more  of  an  infrastructural  problem  in  Korea. 

4.  Limited  ID  for  Individual  students 

For  this  project,  KEDI  provided  two  internet  IDs,  one  for  a  teacher  and  another  for  students. 
I#EARN  distributed  one  ID  for  each  institute.  Thus,  students  write  and  send  it  to  the  teachers  so  that  the 
teachers  can  handle  the  data.  It  creates  the  situation  where  students  do  not  take  care  of  their  letters  directly. 
It  looses  their  feeling  of  control. 

5.  Technical  training  for  teachers 

For  teachers,  networking  is  very  foreign.  It  requires  some  technical  skills.  More  time  and 
training  are  required  until  they  feel  comfortable  to  use  the  internet. 

6.  Integration  into  the  curriculum 

Internet  networking  is  fun  to  use  but  difficult  to  interweave  the  fun  into  the  existing  curriculum 
as  a  meaningful  instructional  activity. 

7.  Discontinuity  of  project  among  teachers 

Due  to  the  short  turn  around  period  for  teachers  (in  Korea,  every  three  or  four  years,  all  the 
teachers  at  the  public  schools  whose  status  are  public  servants  are  obligated  to  change  schools  to 
prevent  the  stagnation  of  being  in  one  place  too  long  and  experience  and  share  their  specialties  in  different 
settings),  the  project  can  lose  the  continuity  among  participating  partner  schools.  Once  the  teacher  in 
charge  or  the  principal  are  appointed  to  another  school,  it  is  difficult  to  expect  to  keep  the  relationship 
between  the  partner  schools. 

While  internet  implementation  during  the  instruction  is  initially  practiced  and  many  problems  are 
first  noticed,  its  educational  value  is  apparent.  It  motivates  students  and  teachers  very  much  and  brings  the 
real  world  into  the  class,  introducing  international  culture  and  society  in  a  natural  manner.  Both  students 
and  teachers  truly  feel  they  are  living  in  a  global  village. 

Sample  I#EARN  Projects 

I#EARN  has  been  involved  with  many  projects  that  reflect  issues  of  global  interest  and  social  concern. 
Certainly  as  the  internet  communication  expands,  the  list  of  I#EARN  projects  will  expand  as  well.  The 
following  projects  are  a  partial  listing  of  those  completed  since  August  1992. 

•  The  Contemporary,  a  news  magazine 

•  A  Vision,  a  literary  anthology 

•  Water  Pumps  for  Nicaragua 

•  Somalia  Relief 

•  The  Holocaust  Project 

•  The  Heroes  Project,  a  hero  identification  project  for  children 

•  Water  Monitoring  Project 

•  First  100  Days  of  the  Clinton  Administration 

•  Holiday's  Project 

•  A  Current  Events  Seminar 
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Social  Implications 

Many  positive  social  outcomes  ate  likely  to  emerge  from  this  project  and  we  hope  to  see  students 
realize  the  fullest  potential  of  data  retrieval  and  shared  communication  capabilities  of  the  internet.  However, 
wisdom  dictates  that  responsible  educators  anticipate  and  try  to  avoid  potential  social  problems  that  might 
emerge  as  hidden  issues. 

Some  of  the  social  issues  inherent  within  this  project  could  emerge  as  the  project  develops  and  so 
we  will  mention  a  few  of  the  more  obvious  ones  here.  First  the  numbers  of  students  and  teachers  who  can 
participate  are  limited.  Although  the  specific  numbers  vary  from  school  to  school,  the  number  of  students 
who  can  be  involved  at  each  school  ranges  at  about  20-30  students,  who  are  selected  by  the  teachers,  thus 
the  majority  of  students  do  not  have  the  opportunity  to  participate  in  this  project.  Students  are  limited  in 
numbers  because  basically,  this  project  is  not  part  of  the  regular  curriculum. 

Funding  remains  a  problem  at  the  individual  student  level  and  at  the  school  level.  At  this  time, 
individual  students  must  accept  the  responsibility  of  payment  for  any  travel  and  so,  students  are  limited  by 
their  family's  financial  status. 

Schools  have  limited  money  for  the  telephone  lines  and  so  the  lines  are  used  for  other  purposes  in 
addition  to  the  internet  project.  Because  teachers  have  difficulty  accessing  the  telephone  lines  during  the 
school  day  when  they  are  needed  for  voice  communication,  the  teachers  find  themselves  extending  their 
work  day  into  the  evening  hours  to  work  when  no  one  else  needs  the  telephone  line.  Although  some 
schools  could  obtain  less  expensive  access  to  the  internet  through  local  universities,  training  then  becomes 
an  issue.  Currently  KEDI  provides  training  on  a  common  access  system,  but  it  schools  choose  to  access 
the  internet  through  a  local  university,  then  system  training  would  need  to  be  offered  by  that  university. 

Internet  access  offers  communication  opportunities  that  have  mixed  results.  There  is  the  potential 
of  emerging  personal  relationships  which  could  have  positive  or  negative  effects.  Also,  there  are  concerns 
about  access  to  pornographic  material  for  students  who  use  the  internet.  Questions  have  emerged  about 
whether  the  balance  of  males  and  females  involved  with  the  project  will  be  representative  of  the  student 
population  at  each  school.  Ethical  considerations  must  be  addressed  concerning  the  responsibility  of  the 
teacher,  the  safety  of  students,  and  the  potential  of  virtual  relationships  to  develop  productively.  In  general, 
teachers  need  to  make  some  judgments  about  the  wisdom  and  guidance  necessary  for  unrestricted  access  to 
the  internet  resources. 

KANSAS  EFFORTS 

Internet  communication  in  Kansas  schools  has  been  led  by  two  driving  forces,  the  individual 
teachers  and  funding.  Like  the  computing  efforts  of  the  past  decade  (Knupfer,  1 993),  internet 
telecommunications  is  most  likely  to  occur  where  interested  teachers  make  a  grassroots  effort  to  gain  access 
to  the  resource. 

Information  Network  of  Kansas  (INK) 

The  Information  Network  of  Kansas  (INK)  was  activated  in  1992  in  an  effort  to  provide  on-line 
services  to  people  throughout  Kansas.  Within  the  INK  network  there  is  information  concerning  legislative, 
banking,  legal,  insurance,  business  and  commerce,  library,  State  agency  services,  local  government, 
electronic  mail,  children's  services,  and  customer  utilities.  Initially,  access  to  INK  was  available  to  schools 
for  an  annual  subscription  fee  of  $50.00  plus  a  connect  time  charge  of  $.40  per  minute  after  the  first 
minute.  Now  the  annual  subscription  fee  has  jumped  to  $180  per  year  plus  aline  charge  of  $7.20  per  hour 
of  use,  plus  the  long  distance  connect  time  charge.  If  a  school  has  access  to  a  computer,  modem, 
telecommunications  software,  and  a  telephone  line,  then  it  can  access  INK. 

Unfortunately,  the  funding  could  not  keep  pace  with  the  huge  on-line  charges  that  began 
accumulating  as  more  and  more  schools  began  to  use  the  services  at  ever-increasing  levels.  For  many  rural 
districts  who  were  not  within  access  to  an  internet  hub  by  through  a  local  telephone  call,  the  INK  became  a 
primary  resource.  And,  rather  than  que  files  and  set  them  up  to  run  in  batch  mode  to  minimize  connection 
time,  schools  found  their  usage  did  not  conform  to  that  option.  Instead,  students  and  teachers  were 
performing  on-line,  interactive  database  searches  that  required  lengthy  connections.  Now  schools  are  finding 
the  accumulating  cost  to  be  a  burden  and  school  usage  of  INK  is  in  jeopardy. 
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Link.  19 

Link  19  is  another  on-line  service  that  is  free  to  Instructional  Television  (ITV)  members, 
beginning  Fall  1994.  Link  19  is  a  powerful  computer-based  information  and  communications  network  that 
provides  access  to  educational  resources,  including  the  internet,  bulletin  boards,  newsgroups,  discussion 
centers,  and  Sharenet.  Those  school  districts  with  full  subscriptions  to  ITV  service  qualify  for  three  free 
connections  to  Link  19.  Less  than  full,  regular  subscription  to  ITV  would  make  a  district  eligible  for  one 
connection  at  most  within  the  district.  Extra  accounts  can  be  purchased  at  an  additional  charge.  At  first, 
Link  19  offered  one  hour  free  access  per  day  for  each  school,  then  that  was  decreased  to  one  hour  per  day  for 
three  sites  per  school  district,  and  currently  the  policy  seems  to  be  fluctuating  down  and  up. 

American  On-line 

Many  Kansas  schools  are  finding  that  American  On-Line  (AOL)  is  one  of  the  better  commercial 
services  available  for  those  who  have  a  local  number  to  call.  Unfortunately  the  many  rural  schools  in 
Kansas  do  not  have  access  to  an  AOL  hub  by  a  local  telephone  number.  The  cost  of  AOL  itself  is  $9.95 
per  month,  plus  any  line  charge  for  long  distance  calls,  plus  $2.95  per  hour.  AOL  provides  full  internet 
service  plus  a  host  of  special  services. 

Direct  Connection  to  Internet 

Schools  that  wish  to  have  a  direct  connection  to  the  internet  will  pay  a  minimum  of  $5,800  per 
year  for  the  service,  plus  any  long  distance  fee  to  access  the  nearest  hub.  The  direct  connection  could  be 
beneficial  to  some  schools,  while  others  would  find  the  additional  fees  overwhelming. 

DISCUSSION 

Financial  difficulties  that  result  in  limited  access  to  software  and  service  providers  have  been  the 
most  difficult  problem  for  Kansas  schools  in  the  telecommunications  effort.  Finding  the  right  provider 
with  appropriate  and  affordable  services  is  difficult  and  so  schools  are  struggling  to  find  funding  for  the 
services.  It  is  possible  that  centralized  control  over  pricing  and  access  that  is  negotiated  for  all  Kansas 
schools  could  help,  but  currently  local  control  means  a  variety  of  situations  and  inequitable  opportunities, 
based  upon  the  local  service  provider's  policies,  the  schools  ability  to  afford  the  service,  and  the  schools 
distance  from  a  local  telephone  call  to  the  internet. 

Although  all  schools  could  potentially  have  access  to  the  internet  and  communicate  globally,  the 
reality  is  that  the  rural  districts  cannot  afford  such  activity  on  a  regular  basis  due  to  the  long  distance 
telephone  charges.  Schools  that  are  within  local  calling  distance  from  a  main  AOL  hub  or  a  university 
internet  connection  have  better  access  at  a  far  reduced  price.  Many  rural  teachers  are  investigating  the 
potential  of  getting  sponsorship  support  from  local  businesses,  but  that  is  not  an  easy  commitment  to 
secure  without  definite  cost  projections  on  a  long-term  basis.  So  many  students  will  continue  to  wait  for 
additional  funding  in  order  to  join  the  internet  community.  One  key  difference  between  Korean  schools  and 
Kansas  schools  is  the  provision  of  the  internet  ID  by  a  central  source  in  Korea. 

Of  course,  the  continuing  problems  that  have  plagued  teachers  during  the  course  of  the 
technological  revolution  continue  as  computing  moves  into  the  telecommunications  and  internet  arena. 
Two  pressing  problems  reported  previously,  shortage  of  training  and  time  (Knupfer,  1989-90)  continue  as 
concerns.  The  internet  presents  an  area  of  access  to  a  wide  variety  of  resources  and  teachers  will  need  some 
training  and  preparation  time  to  adequately  attend  to  this  resource  in  a  practical  and  meaningful  way.  Two 
excellent  sources  of  information  for  teachers  about  how  to  implement  internet  telecommunications  within 
the  classroom  are  TeleSensations:  The  Educators'  Handbook  to  Instructional  Computing,  by  Andres,  Jacks, 
and  Rogers  (1991  and  Integrating  Telecommunications  into  Education,  by  Roberts,  Blakeslee,  Brown,  and 
Lenk(1990). 

If  Kansas  schools  are  to  communicate  with  Korean  schools,  a  third  problem  that  they  might 
encounter  is  language.  Although  many  students  study  second  languages  in  school,  Korean  is  a  language 
that  is  rarely  offered  within  a  public  school  setting.  Foreign  language  classes  have  suffered  as  the  lack  of 
qualified  teachers  and  funding  have  combined  to  create  a  shortage  of  on-site  language  courses.  Schools  in 
Kansas  are  limiting  their  choices  of  foreign  language  to  those  in  greatest  demand,  usually  Spanish,  and  are 
offering  those  classes  increasingly  via  distance  education.  This  situation  makes  it  highly  unlikely  that 
additional  foreign  language  courses  will  be  offered  unless  something  happens  to  change  the  perceived  need 
for  foreign  language  with  respect  to  the  rest  of  the  curriculum.  One  could  speculate  that  schools  should 


a 

ERIC 


329 


31 


place  more  emphasis  on  foreign  language  as  the  improved  communications  promotes  world  wide 
communication.  However,  as  we  can  see  from  the  Australian  and  Korean  experience,  many  countries  offer 
English  as  a  second  language;  this  of  course,  makes  English  the  easiest  route  for  schools  in  the  United 
States  and  does  not  apply  pressure  to  reconsider  decisions  along  that  line. 

,         In  Kansas,  there  are  many  teachers  and  students  who  are  eager  to  join  in  the  telecommunications 
with  Korean  schools.  However,  they  struggle  with  the  ability  to  do  so.  If  Kansas  students  are  to  keep 
abreast  of  the  changes  in  education,  then  they  will  need  to  be  allowed  to  participate  in  activities  beyond  the 
scope  of  the  existing  curriculum  within  the  existing  school  structure.  Liberating  the  students  to  access  the 
internet  will  provide  a  world  of  opportunity  for  them  that  will  only  begin  with  the  international  school 
communication. 

The  potential  of  the  internet  communications  for  shared  projects  among  these  students  in  different  countries 
is  enormous.  Naturally,  schools  face  social  and  cultural  implications  that  result  because  of  access  to  the 
internet  and  its  potential  to  change  the  educational  system,  whether  communication  goes  beyond  the 
boundaries  of  a  country  or  simply  beyond  the  boundaries  of  a  school.  The  internet  truly  has  the  ability  to 
empower  teachers  and  students  to  strike  out  beyond  the  traditional  curriculum  to  change  the  very  nature  of 
their  topics  of  study,  the  methods  by  which  they  study  them,  and  the  result  of  the  educational  process. 


REFERENCES 

Andres,  Y.  M.,  Jacks,  M.,  &  Rogers,  A.  (1991 ).  TeleSensations:  The  educators'  handbook  to  instructional 
computing.  Bonita,  CA:  The  FrEdMail  Foundation. 

Knupfer,  N.  N.  (1993).  Teachers  and  Educational  Computing:  Changing  Roles  and  Changing  Pedagogy.  InR. 
Muffoletto  and  N.  N.  Knupfer  (Eds.),  Educational  Computing:  Social  Perspectives  (pp.  163-179). 
Cresskill,  NJ:  Hampton  Press. 

Knupfer,  N.  N.  (1989-90)  The  Teacher  as  a  Critical  Component  of  Computer  Education  and  School 
Change,  Journal  of  Computing  in  Teacher  Education,  6  (2),  Winter. 

Lee,  Okhwa  (1994,  July).  The  Implications  of  Technology  for  Education.  Korea  Information  Science 
Society  Review.   12  (6). 

Roberts,  N.,  Blakeslee,  G.,  Brown,  M.,  &  Lenk,  C.  (1990).  Integrating  telecommunications  into 
education.  Englewood  Cliffs,  NJ:  Prentice-Hall. 


ERIC 


3*1 


330 


APPENDIX 


KOREAN  SCHOOLS 

KEDI  ID 

IGC  ID 

PARTNER  SCHOOL 

IGC  ID 

1.  Pohang  East  Elem. 
Mr.  Jaekon  Park 

poet 
poes 

pohangeast 

Sunnyside  Elementary 

sunnyside 

2.  Pohang  West  Elem. 
Mr.  Pankun  Kim 

powt 
pows 

pohangwest 

Kimball  Elementary 

kimball 

3.  Ahyun  Middle 
Mr.  Sujong  Lee 

ahyunt 
ahyuns 

ahyun 

Castle  Hill  High  (Au) 
Covington  Jr  High 

castlehs 
covington 

4.  Yangjeung  High 
Mr.  Yongkwon  Cho 

yangt 
yangs 

yangjeung 

Capital  High 

McKinnon  Secon  Col(Au) 

capitalhs 
mckinnon 

5.  Kunkuk  High 
Mr.  Sangul  Lee 

kunt 
kuns 

kungkook 

South  Bronx  High 

sbhs 

6.  Suhgui  High 
Mr.  Kitae  Pyun 

suhguit 
suhguis 

suhguinong 

Kaimuki  High 

7.  Junju  High 
Mr.  Unkyun  Paek 

junjut 
junjus 

junju 

Sammamish  High 

sammamishhs 

8.  KEDI 

ohlee 

kedi 

Dr.  Okhwa  Lee 

♦All  IGC  ID  requires  ©igc.apc.org  to  be  the  full  name  except  the  school  of  Australia. 
Aussi  schools  have  @peg.pegasus.oz.au 

*A11  KEDI  ID  requires  @ns.kedi.re.kr  Eafter  the  account  for  the  full  name. 
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INTRODUCTION 

The  call  for  more  authentic  communication  in  the  language  classroom  has  led  to  an  increasing  use 
of  technology.  Although  technology  is  a  good  tool  to  teach  language  and  culture,  it  is  not  yet  commonly 
used  by  foreign  language  instructors. 

This  article  reports  on  how  foreign  language  instructors  at  Arizona  State  University  at  Tempe  are 
encouraged  and  taught  to  use  technology  in  their  teaching.  Section  one  introduces  the  reasons  why 
technology  is  helpful  for  foreign  language  instruction.  Section  two  describes  foreign  language  instructors' 
attitude  toward  using  technology  in  their  teaching.  Section  three  depicts  the  efforts  of  the  Foreign 
Language  Department  at  Arizona  State  University  to  improve  its  teaching  via  technology.  In  this  section, 
the  renovated  labs  and  a  two-week  language  technology  workshop  are  described  as  well  as  a  study  about  the 
instructors'  attitude  toward  using  technologies  and  their  self-assessment  of  some  technologies.  Finally, 
further  research  topics  are  suggested. 

TECHNOLOGIES  TO  FOREIGN  LANGUAGE  TEACHING 

A  common  complaint  of  foreign  language  students  is  that  they  can  not  communicate  with 
native  speakers  after  they  have  studied  the  language  for  several  years.  During  the  last  decades,  the 
Grammar  Translation  teaching  method,  which  stresses  students'  spelling,  reading,  translating,  and 
writing  ability,  has  been  replaced  by  the  communicative  method  of  language  teaching,  which 
considers  oral  communication  as  the  ultimate  goal  of  language  teaching  (Celce-Murcia  & 
Mcintosh,  1984;  Long  &  Richards,  1987).  This  approach  emphasizes  that  the  language  that  is 
taught  is  of  a  functional  nature,  spoken  in  a  natural  and  informal  manner  (Slaton,  1989). 

Instructors  using  the  communicative  method  are  preparing  learners  to  survive  in  the  target 
language  environment;  therefore,  students  need  to  be  exposed  to  authentic  materials  in  order  to  be 
able  to  react  appropriately  in  real  situations.  Technologies  make  this  possible;  videotape  and 
videodisc  enable  learners  to  listen  to  the  authentic  language  that  is  used  in  real  situations.  Also, 
they  allow  learners  to  be  exposed  to  the  verbal  and  non-verbal  behaviors  of  the  foreign  culture, 
such  as  greeting  (Schneider,  1982).  Saint-Leon  (1988)  indicated  that  videodisc  is  a  good  way  to 
introduce  culture.  Computer  networks,  such  as  E-mail,  enable  learners  to  have  direct  contact  with 
native  speakers  by  writing  (Sayers,  1987)  and  videoconferencing  encourages  the  oral 
communication  between  foreign  language  students  and  target  language  speakers  (Herbst  & 
Wiesner,  1988). 

Besides  the  goal  of  the  instruction,  the  linguistic  proficiency  of  a  teacher  explains  why 
technology  is  beneficial  for  foreign  language  teaching.  If  a  teacher's  speech  is  really  fluent, 
grammatically  correct,  and  untainted  by  a  foreign  accent,  then  the  student  is  truly  fortunate 
(Schneider,  1982);  however,  in  reality,  very  few  teachers  are  so  competent.  Then  native  speakers' 
language  which  can  be  found  on  audio  tapes,  videotapes,  and  videodisks  would  be  advantageous. 

Moreover,  the  pedagogical  proficiency  of  a  foreign  language  teacher  is  limited  to  certain 
extent  no  matter  how  experienced  the  teacher  is  (Schneider,  1982).  S/he  might  go  too  quick  or  too 
slow.  However,  the  computer-assisted  instruction  (CAI)  allows  students  to  learn  a  language  at 
their  own  pace.  Furthermore,  because  different  teachers  have  different  criteria  for  students'  answers 
and  different  attitudes  toward  students'  errors,  the  students'  learning  can  be  affected.  Technology 
allows  a  learner's  mistakes  to  be  a  secret  between  the  learner  and  the  computer.  Thus,  learners  are 
not  discouraged. 

The  levels  of  students  also  explain  why  technology  is  helpful  in  foreign  language 
teaching  (Schneider,  1982).  It  happens  frequently  that  in  a  class  the  ability  of  some  students  is 
high  and  that  of  others  is  low.  All  students  can  benefit  from  technology-based  instruction  that  is 
interactive  and  individualized. 

From  the  above,  we  see  that,  considering  the  instructional  goal,  teachers'  proficiency,  and 
students'  competence  levels,  technology  is  useful  in  foreign  language  instruction. 

FOREIGN  LANGUAGE  INSTRUCTORS'  ATTITUDE 
TOWARD  USING  TECHNOLOGY 

foreign  language  instructors  are  in  favor  of  using  technology  in  their  teaching  because  technology 
provides  authentic  teaching  materials  and  allows  learners  to  be  exposed  more  fully  to  cultures  of  target 
languages  (Hill,  1991;  Herbst  &  Wiesner,  1992).  Nevertheless,  currently  few  instructors  use  technology  in 
their  teaching.  They  are  afraid  because  they  are  not  familiar  with  it.  They  also  do  not  have  enough  access 
to  equipment. 


Suki  Heath  conducted  a  survey  to  find  out  what  teachers  thought  about  authentic 
(satellite)  television,  and  how  widely  it  was  being  used  (Hill,  199 1).  Ninety-three  percent  of  the 
respondents  stated  that  they  would  like  to  use  authentic  television  more  than  they  do  at  present. 
One  striking  theme  to  emerge  from  the  survey  was  "the  almost  unanimous  enthusiasm  for 
authentic  television"  (p.  10).  However,  "Lack  of  available  material,  inadequate  access  to 
equipment . .  .  were  the  most  frequently  quoted  constraints"  (p.  10). 

Saint-Leon  (1988)  found  that ".  . .  using  authentic  materials  requires  that  the  teaching  be 
adjusted  to  the  materials  rather  than  the  materials  to  the  teaching.  Such  an  approach  goes  against 
tradition  in  many  ways  and  therefore  requires  considerable  adjustment"  (p.  38).  She  also  pointed 
out  that  instructors  often  have  difficulty  integrating  the  video  programs  into  the  curriculum. 

From  the  above,  we  see  that  foreign  language  instructors  have  a  positive  attitude  toward  using 
technology  in  their  teaching.  Nevertheless,  they  need  to  be  provided  with  the  necessary  equipment  and  have 
to  be  trained  how  to  use  technology  effectively  and  how  to  integrate  it  into  their  instruction. 

THE  REFORMATION  AT  ARIZONA  STATE  UNIVERSITY 

Due  to  the  need  to  integrate  technology  with  language  teaching,  the  Foreign  Language  Department 
at  Arizona  State  University  renovated  the  language  learning  lab  in  1992.  The  new  labs  were  effective  in 
spring  of  1994.  In  summer  of  1994,  a  technology  language  workshop  was  held  to  train  the  instructors  in 
this  department. 

The  Renovated  Labs 

The  new  language  learning  technologies  include  an  audio-visual  lab,  a  language  computing  lab, 
and  six  mediated  classrooms.  These  technologies  not  only  allow  students  to  learn  languages  by  using 
multimedia  software,  but  also  enable  instructors  to  develop  multimedia  programs. 

The  Audio-Visual  Lab 

Fifty-six  booths  are  in  this  lab  and  each  booth  has  an  audio  control  panel  and  a  video  monitor. 
Students'  cassette  tape  recorders  are  in  a  separate  room  where  the  instructors  can  simultaneously  duplicate  56 
cassette  tapes  at  high  speed.  Before  class,  an  instructor  can  quickly  download  material  into  the  students' 
tapes.  The  audio  control  panel  in  each  booth  allows  the  student  to  control  his  (her)  tape.  Of  course,  the 
instructor  can  control  the  students'  activities  whenever  he  (she)  wants  to. 

A  student  can  learn  a  language  not  only  by  listening,  but  also  can  learn  the  culture  by  watching  a 
video  on  a  video  monitor.  There  is  a  "hot  seat"  among  every  four  students  in  the  same  row.  The  student 
on  the  "hot  seat"  has  a  video  control  panel  that  allows  him  (her)  to  control  the  video  images  on  the 
monitors  of  the  four  students.  S/he  can  freely  rewind  a  videotape  and  have  a  small  group  discussion  with 
the  other  three  learners. 

The  Language  Computing  Lab 

Fifty-six  computers  locate  in  this  lab  and  each  computer  has  a  headphone  and  a  CD-ROM  drive. 
Students  can  view  multimedia  programs,  write  E-mail,  and  search  for  information  in  Internet  via  Gopher, 
Mosaic  and  Netscape.  Toolbook  packages  for  learning  different  languages  are  available  in  this  lab.  The 
equipment  here  also  allows  instructors  to  design  and  produce  their  own  instruction. 

A  special  equipment,  the  TECH  Commander,  allows  an  instructor  to  view  the  screen  of  an 
individual  student  and  take  control  of  the  screen.  Thus,  the  teacher  and  the  learner  can  watch  the  same 
screen  and  discuss  what  is  on  the  screen  by  headphones.  The  instructor  can  also  let  students  view  what  is 
on  his/er  screen  and  provide  instruction.  A  computer  with-a  video  overlay  card  and  the  TECH  Commander 
enables  all  students  to  watch  a  videotape  program  on  their  computer  screens.  A  special  software  installed  in 
some  computers  allows  students  with  visual  problems  to  enlarge  letters  on  their  monitors,  and  some 
hardware  also  is  provided  for  hearing-impaired  learners. 

The  Mediated  Classrooms 

A  mediated  classroom  consists  of  a  multimedia  computer,  a  video  monitor,  and  a  video  projector. 
The  Rauland-Borg  Ranger  Media  Management  System  can  distribute  video  and  multimedia-computer 
capabilities  to  this  classroom  (on  the  second  floor)  from  a  control  room  (in  the  basement)  where  all  material 
sources  are  stored.  Thus,  an  instructor  does  not  have  to  bring  those  sources  (videotapes,  laser  discs,  or 
CDs)  to  class,  but  s/he  can  control  these  materials.  The  other  six  mediated  classrooms  are  about  to  be 
completed  and  all  presentation  materials  will  be  generated  and  distributed  from  the  control  room  by 
computer  programs. 
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The  Language  Technology  Workshop 
Description 

In  addition  to  the  renovation  of  the  labs  and  classrooms,  the  department  also  held  a  workshop  to 
train  instructors  to  use  these  technologies  in  their  teaching.  The  workshop  was  held  by  Dr.  Dan  Brink  right 
after  the  semester  ended,  from  May  16  to  May  27,  1994.  The  purpose  of  the  two- week  workshop  was  to 
train  the  instructors  in  the  Foreign  Language  Department  to  be  aware  of  the  new  technologies  and  be 
willing  to  use  them  in  their  teaching.  Instructors  in  the  department  were  not  required  to  t--Jce  part  in  the 
workshop,  but  encouraged  to  attend,  and  coordinators  of  language  sections  were  strongly  recommended  to 
participate  in  it. 

Every  day,  the  workshop  began  with  an  orientation  meeting,  followed  by  presentations  and 
discussion  from  9:00  a.m.  to  noon,  and  hands-on  practice  from  1:30  p.m.  to  4:30  p.m.  in  the  computer  lab. 
In  the  afternoon,  the  instructors  defined  themselves  as  beginners  or  advanced  learners  and  received  different 
tasks  which  were  related  to  what  they  learned  in  the  morning.  During  the  afternoon,  Dr.  Brink  and  two 
teaching  assistants  helped  the  instructors  individually. 

Toolbook  and  E-mail  were  the  main  focus  of  the  workshop.  Toolbook,  like  HyperCard,  is  an 
authoring  program  which  allows  instructors  to  design  instruction.  On  the  first  morning,  the  presenter  (Dr. 
Brink)  introduced  the  workshop,  the  language  learning  labs  and  classrooms,  Toolbook  basics,  and  Internet 
basics.  On  the  second  day,  the  participants  learned  to  operate  the  equipment  in  the  mediated  classroom,  E- 
mail  basics,  and  Toolbook  primer.  More  E-mail  and  Toolbook  basics  were  taught  at  the  third  session.  On 
the  fourth  morning,  the  participants  listened  to  a  talk  about  using  E-mail  in  Spanish  classes,  discussed  their 
homework,  and  learned  importing  and  screen  design.  On  the  last  day  of  the  first  week,  the  participants 
watched  some  sample  programs,  learned  about  the  Internet,  and  experienced  multimedia. 

The  second  week  started  with  designing  flowcharts  and  discussing  a  Toolbook  project  to  teach 
passive  voice.  In  the  seventh  session,  a  sample  project  of  teaching  French  was  demonstrated  and  getting 
sound  into  Toolbook  was  introduced.  On  the  eighth  morning,  the  participants  got  acquainted  with  the 
"humanistic"  approach  of  language  instruction  and  computer  conferencing.  The  next  day  was  devoted  to 
colors,  instructional  design  models,  and  sound  clips.  On  the  last  day,  some  useful  features  of  Toolbook 
such  as  scripts  were  introduced. 

Participants 

There  were  twelve  participants  in  the  workshop:  one  German  professor,  two  Italian  professors,  two 
Spanish  professors,  four  French  professors,  two  ESL  (English  as  a  Second  Language)  professors,  and  one 
English  Linguistic  professor.  Most  of  them  are  coordinators  of  different  language  sections  and  can 
considerably  influence  other  instructors.  They  are  experienced  language  professors;  al!  but  two  have  taught 
language  for  more  than  10  years  and  five  of  them  for  even  more  than  25  years.  Five  of  the  twelve 
participants  attended  the  workshop  for  one  week  and  seven  of  them  stayed  for  two  weeks. 

Getting  acquainted  with  technology,  improving  teaching  techniques,  learning  to  develop  language 
software,  and  incorporating  technology  into  language  teaching  were  the  instructors'  motivation  to  attend  the 
workshop. 

Methodology  of  the  Study 

Four  questionnaires  were  distributed  to  the  participants  before,  at  the  end  of  the  first  week,  and  at 
the  end  of  the  second  week  of  the  workshop.  The  first  and  second  questionnaires,  which  were  filled  out  by 
the  participants  before  the  workshop,  were  designed  to  check  their  technology  background  and  attitude 
toward  using  technology  in  language  teaching  (see  Questionnaire  I  &  II  in  Appendix).  The  technologies 
mentioned  in  the  questionnaires  include  E-Mail,  computers,  digital  technology  and  analog  technology.  The 
participants  ranked  from  one  to  five  the  strength  of  their  attitude,  one  for  the  lowest  and  five  for  the  highest. 

At  the  end  of  the  first  week,  the  participants  answered  the  third  questionnaire  that  was  to 
investigate  their  self-assessment  of  their  learning  in  the  first  week.  Their  Toolbook  and  E-mail  basics' 
familiarity,  attitude  of  using  these  technologies  in  foreign  language  teaching,  and  feelings  of  using  them  in 
their  own  teaching  were  studied  (see  Questionnaire  III  in  Appendix). 

At  the  end  of  the  workshop,  the  seven  participants  who  stayed  for  the  second  week  filled  in  the 
fourth  questionnaire.  This  questionnaire  was  not  only  used  to  look  at  the  self-assessment  of  their  learning, 
but  also  used  to  compare  their  attitude  toward  using  technologies  in  teaching  before  and  after  the  workshop 
(see  Questionnaire  IV  in  Appendix). 

The  ranking  numbers  were  averaged  (see  Tables  in  Appendix).  The  difference  between  the  average 
of  the  first  week  and  the  second  week  was  investigated.  The  tables  also  show  the  average  of  all  respondents 
and  the  average  of  the  participants  who  stayed  for  the  second  week. 
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Results  &  Discussion 

The  responses  of  the  first  questionnaire  revealed  that  most  of  the  participants  vvrd  not  familiar 
with  the  technology.  Six  of  them  (55%)  knew  nothing  about  E-Mail;  four  of  them  (36%)  knew  some  E- 
Mail  basics;  only  one  of  them  (9%)  was  familiar  with  different  systems  of  E-Mail.  As  to  their  general 
computer  background,  five  of  them  (45%)  did  not  know  anything  about  computers;  four  of  them  (36%) 
knew  some  basics  about  Windows  or  Mac;  one  (9%)  was  well-versed  with  various  applications,  and  one 
(9%)  considered  himself  as  a  programmer.  One  professor  did  not  respond  to  the  questions. 

The  responses  also  showed  that  technology  was  not  heavily  used  by  the  instructors.  None  of  them 
used  E-Mail  in  their  teaching.  Only  one  of  them  had  studied  instructional  design  or  ever  used  digital 
technology;  three  of  them  employed  analog  technology  in  their  teaching. 

Although  the  participants  were  not  familiar  with  technology  and  also  did  not  use  technology  in 
their  teaching  significantly,  their  attitude  toward  using  technology  in  language  teaching  was  very  positive. 
Table  1  in  Appendix  indicates  that  the  participants  ranked  their  attitude  above  average,  when  asked  what 
they  thought  of  using  the  technologies  in  language  teaching  before  the  workshop. 

One  professor  responded,  "It  is  necessary  for  the  teacher  to  use  it  or  to  be  old  fashioned.  We  are  in 
era  of  the  technology."  Another  participant  stated,  "I  think  it  can  be  very  useful,  even  though  I  probably  do 
not  completely  understand  or  know  its  full  potential."  Another  professor  answered,  "I  am  anxious  to 
incorporate  this  technology  in  my  teaching.  I  think  it  will  be  extremely  helpful." 

Some  professors  preferred  digital  technology  to  analog  technology  because  they  could  have  access 
to  images  easily  and  obtain  better  quality  images.  One  professor  mentioned,  "I  prefer  digital  [technology] 
because  I  like  something  clear  and  neat."  Another  professor  also  expressed,  "Sound  quality  [of  the  digital 
technology]  is  superior  to  [that  of  the]  analog. ...  I  get  tired  of  cleaning  tape  recorder  heads  and  still 
having  bad  quality,  but  it  is  easier  to  copy."  Low  cost  and  easy  access  are  advantages  of  analog  technology. 
A  professor  mentioned  that  analog  technology  is  "cheap  and  readily  available,"  but  finding  the  exact 
segments  s/he  wanted  is  difficult. 

The  responses  of  the  third  questionnaire,  which  was  filled  out  by  the  participants  at  the  end  of  the 
first  week,  revealed  that  the  instructors  were  not  very  confident  about  the  knowledge  they  learned  during  that 
week.  The  average  of  Toolbook  basics  was  3.2  and  the  one  of  E-mail  was  only  2.8  (see  Table  2  in 
Appendix).  Nevertheless,  the  instructors'  attitude  about  using  the  technologies  in  teaching  was  not  affected 
by  their  knowledge  about  them.  They  had  very  positive  attitude  toward  using  Toolbook  in  language 
teaching  and  thought  that  they  would  use  it  in  their  own  teaching. 

The  attitude  toward  using  E-Mail  was  not  as  positive  as  the  one  toward  using  Toolbook  because 
the  subjects  and  levels  that  the  participants  teach  would  affect  the  instructors  to  use  this  technology.  The 
linguistics  professor  said,  "I'm  not  convinced  that  all-English-speaking  classes  in  linguistics  would 
benefit."  Another  professor  stated,  "It  can  be  useful  for  some  high  classes."  The  professors  who  ranked  it 
high  thought  that  E-Mail  allows  learners  to  be  exposed  to  cultures  when  students  write  to  native  speakers. 

Seven  professors  stayed  at  the  workshop  in  the  second  week.  All  of  the  average  numbers  in  the 
third  column  of  table  2  were  higher  than  the  ones  in  the  first-week  columns  (see  Table  2  in  Appendix). 
These  results  indicate  that  these  instructors  felt  more  confident  on  their  knowledge  about  Toolbook  and  E- 
mail  at  the  end  of  the  second  week  than  at  the  end  of  the  first  week.  The  Toolbook  familiarity  average  rose 
slightly  frc    3.0  to  3.4  and  the  average  of  the  attitude  also  rose  from  4.3  to  4.5.  All  of  the  respondents 
thought  that  they  would  use  Toolbook  in  their  own  teaching  at  the  end  of  the  second  week.  The  E-mail 
familiarity  average  jumped  from  2.7  to  3.9,  and  the  average  of  the  attitude  also  rose  from  3.9  to  4.3.  The 
average  of  using  E-mail  in  their  own  teaching  went  from  3.4  up  to  3.9. 

The  results  in  Table  1  indicate  that  the  instructors'  general  attitude  toward  using  these  technologies 
in  language  teaching  after  the  workshop  was  not  much  different  from  the  one  before  the  workshop.  The 
average  of  the  instructors'  attitude  toward  using  E-mail  in  teaching  slightly  went  from  4.0  to  4.3.  Their 
average  response  to  digital  technology  and  analog  technology  before  and  after  the  workshop  was  about  the 
same.  The  average  of  their  attitude  toward  using  computers  dropped  slightly  from  4.5  down  to  4.3.  Since 
the  difference  was  due  to  only  one  participant's  response,  the  change  was  not  considered  significant.  The 
researcher  thought  that  the  unchanged  post-workshop  attitude  was  due  to  the  strong  positive  attitude  the 
instructors  had  before  the  workshop.  The  majority  of  the  participants  thought  that  the  workshop  was 
helpful  and  similar  workshops  were  needed  in  the  future. 
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CONCLUSION 

From  the  stu^v\  we  can  tell  that  the  foreign  language  instructors  were  not  familiar  with  technology 
and  seldom  use  technology  in  their  teaching  before  the  workshop.  However,  they  had  very  positive  attitude 
toward  using  technology  in  language  teaching.  During  the  workshop,  they  gained  general  knowledge  of 
technologies,  such  as  Toolbook,  Internet,  instruction  design,  and  multimedia.  At  the  end  of  the  workshop, 
they  felt  more  comfortable  about  technology  and  had  similar  positive  attitude  toward  using  it  in  instruction. 

Since  the  participants  had  very  positive  attitude  toward  technology  before  they  attended  the 
workshop,  the  study  did  not  reveal  the  relationship  between  the  participants'  knowledge  and  attitude.  It 
would  be  helpful  for  researchers  to  look  at  the  correlation  between  instructors'  technology  knowledge  and 
their  attitude.  To  investigate  the  instructors'  knowledge  and  attitude  one  year  later  would  be  interesting, 
too. 

There  were  only  twelve  participants  in  this  study.  It  would  be  beneficial  to  conduct  research  on 
more  foreign  language  instructors  to  obtain  a  broader  picture  of  foreign  language  instructors'  familiarity  of 
technology  and  their  attitude  toward  using  technology  in  teaching.  More  instructors'  technology  training 
programs  need  to  be  designed. 
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Questionnaire  1 

Language  Technology  Workshop 
Participant  Questionnaire: 
Name:  


E-Mail  address:  

Department,  mailcode: . 


Self-assessment  of  E-Mail  familiarity  (circle  one): 

none  E-mail  basics  familiar  with  many  systems  guru 

Have  you  ever  used  E-Mail  in  teaching?  

If  so,  give  a  brief  description: 

Self-assessment  of  computing  familiarity  (circle  one): 

none  windows/Mac  basics        many  applications  programmer 

Have  you  ever  used  a  programming  language?  

If  so,  which  one(s): 

Have  you  ever  used  digital  technology  in  teaching?  

If  so,  describe  what  you  did: 

Have  you  ever  made  significant  use  of  analog  technology  in  teaching  (other  than  visiting  the 

language  lab)?  

If  so,  describe: 

Have  you  ever  studied  instructional  design?  

If  so,  describe: 

Which  of  these  activities-or  which  other  activity  or  activities-would  you  like  to  learn  more  about  in  this  workshop? 
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Questionnaire  2 


Language  Technology  Workshop 
Participant  questionnaire  II: 

Name:   

What  do  you  think  of  using  E-mail  in  foreign  language  teaching?  (circle  one) 

12  3  4  5 

not  useful  very  useful 

Please  describe  your  attitude: 


What  do  you  think  of  using  computers  in  foreign  language  teaching?  (circle  one) 
12  3  4  5 

not  useful  very  useful 

Please  describe  your  attitude: 


What  do  you  think  of  using  digital  technology  in  foreign  language  teaching?  (circle  one) 
12  3  4  5 

not  useful  very  useful 

Please  describe  your  attitude: 


What  do  you  think  of  using  analog  technology  in  foreign  language  teaching?  (circle  one) 
!  2  3  4  5 

not  useful  very  useful 

Please  describe  your  attitude: 
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Questionnaire  3 

Language  Technology  Workshop  Name: . 
Participant  questionnaire  III: 

How  long  have  you  taught  the  language?  _ 


What  is  your  main  teaching  area?  (circle  one) 

language  linguistics  literature  others: 

Please  describe  your  motivation  of  attending  the  workshop: 

Toolbook  and  E-mail  basics  refer  to  what  we  learn  this  week. 
Self  assessment  of  Toolbook  basics  familiarity  (circle  one): 

12  3  4  5 

not  good  very  good 

What  do  you  think  of  using  Toolbook  in  foreign  language  teaching  (circle  one)? 

12  3  4  5 

not  good  very  good 

Are  you  going  to  design  lessons  using  Toolbook  in  your  teaching  (circle  one)? 

12  3  4  5 

not  good  very  good 

Please  explain: 


Self  assessment  of  E-mail  basics  familiarity  (circle  one): 

12  3  4  5 

not  good  very  good 

What  do  you  think  of  using  E-mail  in  foreign  language  teaching  (circle  one)? 

1  2  3  4  5 

not  good  very  good 

Are  you  going  to  use  E-mail  in  your  teaching  (circle  one)? 

1  2  3  4  5 

not  good  very  good 

Please  explain: 
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Questionnaire  4 

Language  Technology  Workshop  Name:  

Participant  questionnaire  IV: 

Toolbook  and  E-mail  basics  refer  to  what  we  learned  these  two  weeks. 

Self  assessment  of  Toolbook  basics  familiarity  (circle  one): 

12  3  4  5 

not  good  very  good 

What  do  you  think  of  using  Toolbook  in  foreign  language  teaching  (circle  one)? 

12  3  4  5 

not  useful  very  useful 

Are  you  going  to  design  lessons  using  Toolbook  in  your  teaching  (circle  one)? 

1  2  3  4  5 

not  at  all  definitely  yes 

Please  explain: 


Self  assessment  of  E-mail  basics  familiarity  (circle  one): 

12  3  4  5 

not  good  very  good 

What  do  you  think  of  using  E-mail  in  foreign  language  teaching  (circle  one)? 

12  3  4  5 

not  useful  very  useful 

Are  you  going  to  use  E-mail  in  your  teaching  (circle  one)? 

12  3  4  5 

not  at  all  definitely  yes 

Please  explain: 


What  do  you  think  of  using  computers  in  foreign  language  teaching?  (circle  one) 
12  3  4  5 

not  useful  very  useful 

What  do  you  think  of  using  digital  technology  in  foreign  language  teaching?  (circle  one) 
12  3  4  5 

not  useful  very  useful 

What  do  you  think  of  using  analog  technology  in  foreign  language  teaching?  (circle  one) 
12  3  4  5 

not  useful  very  useful 

What  do  you  think  of  having  another  language  technology  workshop? 

1  2  3  ~        4  5 

not  necessary  definitely  necessary 

Suggestions  for  a  coming  workshop  (use  the  page  at  the  back  if  necessary): 
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Table  1 

The  Instructors'  Attitude  of  Using  Technologies  before  and  after  the  Workshop 


Before  the  Workshop 


After  Workshop 


E-mail 

Computers 

Digital 
Technology 

Analog 
Technology 


Average  of  all 
Respondents 

4.0 

4.5 

4.6 

4,1 


Average  of  7 
Participants 

4.0 

4.5 

4.7 


Note: 

Average  of  7  participants  =  average  of  the  participants  who  stayed  at 

the  workshop  for  the  second  week 


4.3 
4.3 
4.7 

4.1 


Table  2 

The  Instructors'  Self  Assessment  of  Toolbook  and  E-mail  Familiarity  and  Attitude  of  Using  them  at  the 
End  of  the  First  Week  and  at  the  End  of  the  Second  Week 


At  the  End  of  the  1st  Week  At  the  End  of 

the  2nd  Week 

Average  of  all     Average  of  7 
Respondents  Participants 


Toolbook 

Know 

3.2 

3.0 

3.4 

Attitude 

4.4 

4.3 

4.5 

Will  use 

4.5 

4.4 

5.0 

E-mail 

Know 

2.8 

2.7 

3.9 

Attitude 

3.8 

33 

4.3 

Will  use 

3.4 

3.4 

3.9 

Note: 

Average  of  7  participants  =  average  of  the  participants  who  stayed  at 

the  workshop  for  the  second  week 
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INTRODUCTION 


In  undertaking  the  critical  analysis  of  instructional  design,  we  will  frame  our  discussion  within 
Habermas'  three  fundamental  human  interests,  technical,  practical  and  emancipatory.  The  primary  goal  of 
this  paper  is  to  explore  alternative  approaches  for  instructional  designers'  reflection  and  critique. 
Ultimately,  this  reflection  and  critique  should  shed  light  on  how  learning/instruction  might  be  designed. 

Instructional  design,  as  an  independent  discipline,  thrived  when  Skinner  translated  behavioral 
learning  theory  into  programmed  instruction  in  the  1960s  (AECT,  1977;  Glaser,  1982;  Reiser,  1987). 
Since  then,  it  has  been  aligned  closely  with  learning  psychology,  and  has  been  expanding  its  theoretical 
constructs  by  drawing  upon  communication  and  engineering  theories  (Eraut,  1985;  Reiser,  1987;  Richey, 
1993).  In  the  past  three  decades,  its  theoretical  foundation  became  oriented  toward  different  paradigms. 
Instead  of  looking  at  the  comprehensive  aspects  of  instructional  design,  we  will  focus  on  its  major 
theoretical  foundations. 

Based  on  the  theoretical  analysis  of  instructional  design,  we  will  then  summarize  the  general 
principles  of  the  three  interests.  With  a  deductive  approach,  we  will  draw  implications  from  these 
principles  for  designing  instruction.  In  order  to  derive  these  implications,  several  dimensions  of 
instructional  design  will  be  examined.  They  are  knowledge  construction  process,  and  the  roles  of  and 
relationships  among  resources,  learners,  teachers,  instructional  designers  and  even  the  client.  In  addition, 
the  social,  cultural  and  political  context  and  environment  of  instructional  design  will  also  be  discussed. 


Focus  of  design  -  toward  prescription 

The  discipline  of  instructional  design  began  at  the  period  when  programmed  instruction 
incorporated  systems  thinking  and  communication  theory  in  the  1960s.  In  the  following  years, 
instructional  design  gained  more  and  more  recognition  in  military  and  business  training,  especially  when 
Gagne's  theory  was  prevalently  adopted  by  instructional  designers.  Since  then,  instructional  designers  have 
become  more  and  more  ambitious  about,  and  confident  of,  their  capability  to  make  learning  efficient  and 
effective  by  matching  instructional  methods  with  learning  objectives.  Until  the  time  when  Gagne's  (1985) 
cumulative  learning  theory  was  integrated  with  design  theory,  the  theoretical  foundation  of  instructional 
design,  which  is  heavily  dependent  on  learning  psychology,  was  rarely  challenged. 

Although  Bruner  declared  the  need  to  differentiate  descriptive  learning  theory  from  prescriptive 
instructional  theory  in  the  1960s  (Glaser,  1982;  Clark,  1987),  he  has  not  constructed  any  theoretical 
guidance  for  instructional  design.  However,  Reigeluth's  (1983)  conditions-methods-outcomes  framework, 
which  takes  outcomes  and  conditions  as  givens  and  prescribes  the  best  methods  as  variables  of  interest, 
exemplifies  very  clear  guidelines  for  designing  instruction  at  the  micro-,  mid-  and  macro-levels.  He  affirms 
that  descriptive  instructional  theory  with  an  if-then  outcomes  orientation  can  only  passively  predict  what 
will  happen  by  following  certain  instructional  actions,  whereas  prescriptive  instructional  theory,  with  an  in- 
order-  to/ought-to  methods  orientation,  is  more  rigorous  in  identifying  optimal  instruction.  A  prescriptive 
instructional  theory  with  adaptable  principles,  in  essence,  is  intended  to  provide  practitioners,  especially 
novice  or  incompetent  practitioners,  with  better  guidelines  for  making  better  instructional  choices, 
judgements  and  decisions. 

Landa  (1983)  reaffirms  that  theories  and  programs  of  instruction  cannot  be  directly  derived  from 
theories  and  programs  of  learning  because  1 )  learning  theory  does  not  tell  anything  about  which  of  its 
propositions  should  be  taken  into  account  and  combined  in  order  to  state  an  effective  prescriptive 
instructional  proposition,  2)  the  information  provided  by  learning  theories  (both  descriptive  and 
prescriptive)  and  learning  programs  is  necessary  for  building  an  instructional  theory  and  instructional 
program,  but  not  sufficient,  and  3)  in  many  psychological  and  pedagogical  theories,  regularities  of  learning 
are  viewed  as  inherent  and  independent  of  instructional  influences.  Following  Reigeluth  and  Landa,  Heinich 
( 1984),  Winn  (1986)  and  Clark  (1987,  1989)  also  advocate  the  value  of  prescriptive  instructional  theory  for 
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designing  instruction.  Since  then,  prescriptive  instructional  theory  has  been  regarded  as  a  milestone  which 
makes  an  extraordinary  contribution  to  the  theory  base  of  instructional  design. 

What  distinguishes  prescriptive  from  descriptive  instructional  theory  is  the  spirit  of  pragmatism, 
since  the  former  focuses  on  the  actions  that  could  promote  instructional  efficiency,  effectiveness  and  appeal. 
While  emphasizing  the  need  for  prescriptive  instructional  theories,  Reigeluth  (1993)  also  recognizes  the 
importance  of  learning  theories  to  instructional  design.  He  contends  that  many  instructional  designers,  such 
as  Gagne,  Gropper  and  Scandura,  create  both  learning  theory  and  instructional  theory.  The  link  between  the 
two  has  been  strong.  However,  in  order  to  provide  novice  or  less  competent  teachers/instructional  designers 
with  help  and  guidance,  prescriptive  instructional  theories  will  carry  much  more  weight  than  descriptive 
instructional  theories.  They  are  tools  to  be  applied,  and  especially  to  be  adapted,  to  meet  individual  needs  in 
different  contexts.  Such  adaptation  can  be  achieved  through  formative  evaluation  of  a  prescribed 
instructional  model  or  theory.  In  other  words,  when  a  prescriptive  theory  is  constructed,  it  is  expected  to  be 
constantly  tested  and  revised  in  reality. 

Goal  of  design  -  influence  from  cognitive  psychology 

Even  though  prescriptive  instructional  theory  has  elevated  instructional  design  to  the  state  of  a 
more  independent  field,  its  major  theoretical  foundation  continues  to  be  learning  psychology,  which  is 
rapidly  advancing.  However,  Bonner  (1988)  points  out  that  cognitive  psychologists  do  not  necessarily  find 
the  link  between,  or  synthesis  of,  learning  and  instructional  theory  satisfactory,  despite  instructional 
designers'  interest  in  cognitive  theory  and  their  attempts  to  integrate  it  into  their  theory  and  research.  She 
even  indicates  some  conflicts  between  the  principles  of  cognitive  theory  and  the  practice  of  instructional 
design.  For  example,  instructional  designers  look  for  practical  solutions  to  training  people  efficiently, 
which  means  quickly  and  at  low  cost,  while  cognitive  psychologists  are  concerned  primarily  with  the  study 
of  cognition  and  learning.  Such  disparity  could  be  traced  to  the  pragmatic  emphasis  of  prescriptive 
instructional  theory. 

Winn  (1989)  laments  that  as  long  as  instructional  design  separates  design  from  delivery,  it  still 
operates  at  a  behavioral  mode,  even  though  the  behavioral  objective  and  outcomes  are  changed  to  cognitive 
terms.  He  proposes  that  instructional  designers  should  reflect  more  upon  learning  theory.  Calls  for  closer 
linkage  between  learning  theory  and  instructional  theory  are  now  being  frequently  voiced  (Glaser,  1990; 
Tennyson,  1990;  Hollis,  1991)!  After  years  of  effort  in  establishing  instructional  design  theories  and  in 
applying  them  to  training,  instructional  designers  are  finding  out  that  their  practice  is  lagging  behind  the 
advancement  of  the  sources  upon  which  they  once  drew.  Although  Reigeluth  (1989)  calls  for  enriching  the 
knowledge  base  of  instructional  design,  many  of  the  instructional  design  theories  have  not  yet  been  revised 
to  incorporate  the  new  findings  of  cognitive  learning  theories. 

Unlike  the  1970s,  when  instructional  theorists  successfully  used  the  principles  of  behavioral  learning 
theories  to  generate  instructional  theories,  they  now  have  increased  difficulties  in  using  advances  in 
cognitive  learning  theory  to  generate  related  instructional  principles.  One  of  the  major  reasons  might  be 
that  the  community  of  instructional  theorists  has  been  endeavoring  to  apply  theories  rather  than  developing 
theories  and  consolidating  the  knowledge  base.  Efforts  expended  in  constructing  instructional  theories  based 
on  cognitive  psychology,  such  as  those  to  teach  higher  order  thinking,  are  still  inadequate.  The  theoretical 
foundation  of  instructional  design  is  questioned  again  when  constructivism  attracts  the  attention  of 
instructional  designers. 

Design  for  constructive  learning 

While  instructional  designers  are  still  facing  the  challenge  from  cognitivist  theorists, 
constructivism  proponents  have  arrived  on  the  scene.  Notably,  when  the  three  cognitive  psychologists, 
Collins,  Brown  and  Newman  (1989)  proposed  the  cognitive  apprenticeship  instructional  model,  some 
instructional  designers  started  instilling  the  spirit  of  this  model  into  their  design  of  instructional  materials 
(Duffy  &  Jonasse  i,  1991 ;  Duffy,  Lowcky,  &  Jonassen,  1992)  Some  instructional  theorists  are  not 
content  to  sec  the  distinct  line  drawn  between  descriptive  and  prescriptive  theory  (Bednar,  Cunningham, 
Duffy  and  Perry,  1991;  Duffy  and  Jonassan,  1991)  and  contend  that  instructional  designers  should  design  in 
context,  not  in  procedures  for  teachers  to  follow. 


The  questions  posed  by  cognitivist  theorists  and  constructivism  proponents  are  quite  similar. 
They  both  embrace  a  closer  link  between  learning  theory  and  instructional  theory.  Since  learning  theory 
has  evolved  from  behaviorism  to  cognitivism  to  constructivism  and  even  to  cultural  psychology  (Bruner, 
1990),  instructional  theorists  undoubtedly  will  have  to  struggle  with  such  evolution  while  constructing 
their  own  theoretical  foundations  if  they  hope  to  maintain  instructional  design  as  an  independent  discipline. 

Challenges  from  prescriptive  instruction,  cognitive  psychology  and  constructivism  to  the 
theoretical  foundation  of  instructional  design  certainly  contribute  to  it«  evolution.  However,  looking  into 
the  nature  of  these  challenges  closely,  we  can  find  that  many  of  the  arguments 

about  what  an  instructional  theory  should  be  centers  on  two  questions.  First,  what  are  the  sources  upon 
which  an  instructional  theory  can  draw?  Should  instructional  principles  be  derived  completely  from 
learning  psychology?  The  second  question  is,  will  a  prescriptive,  approach  to  designing  instruction  violate 
the  way  people  leam?  We  regard  the  two  as  separate  questions  that  need  to  be  dealt  with  independently.  By 
doing  yo,  we  can  then  synthesize  the  new  advancement  of  learning  theories  and  include  other  design 
possibilities  with  the  knowledge  base  of  current  instructional  design. 


New  lens  for  design  -  critical  theories 

Unlike  the  local  challenges  from  the  proponents  of  learning  psychology,  critical  theories  seem  to 
provide  some  alternatives  for  instructional  designers  to  reflect  upon  the  possible  ways  for  improvement 
from  a  more  global  view.  In  Paradigms  Regained  (Hlynka  and  Belland,  1991),  a  variety  of  critical 
perspectives  is  introduced  to  the  field  of  educational  technology:  Habermas'  critical  theory,  criticism  and 
connoisseurship,  semiotics,  postmodernism  and  deconstructivism,  and  curriculum  criticism.  Among  these 
diverse  critical  perspectives,  Habermas'  theory  has  been  most  deliberately  applied  to  the  study  of 
instructional  design,  especially  by  Streibel. 

Habermas  (1972)  identifies  three  basic  cognitive  interests:  technical,  practical  and  emancipatory. 
These  interests  constitute  the  three  types  of  science  by  which  knowledge  is  generated  and  organized  in  our 
society.  Grundy  (1987)  presents  a  very  good  interpretation  of  these  three  human  interests.  She  contends 
that  technical  interest,  according  to  Habermas,  is  an  empirical/analytical  way  of  knowing,  with  an  aim  at 
control  and  prediction  through  implementing  pre-existing  ideas.  When  we  apply  it  to  instruction  and 
learnin  >,  we  will  find  that  effectiveness  and  efficiency  are  the  goals  for  such  interest.  According  to 
Haber.nas, 

the  type  of  knowledge  generated  by  empirical-analytic  science  is  grounded  in  experience 

and  observation,  often  produced  through  experimentation.  There,  is  a  relationship 
between  knowledge  and  power  and  between  science  and  technology.  Habermas  is  making 
a  stronger  claim  that  there  is  a  possible  relationship  between  prediction  and  control.  For 
Habermas  the  fundamental  interest  which  guides  empirical-analytic  science  is  an  interest 

in  control  and  the  technical  exploitability  of  knowledge  The  technical  interest  is  a 

fundamental  interest  in  controlling  the  environment  through  rule-following  action  based 
upon  empirically  grounded  laws  (cited  in  Grundy,  p.  10-12). 

Practical  interest,  in  Grundy's  words,  means  historical  and  hermeneutic  ways  of  knowing  in  which 
meaning  is  socially  and  culturally  constructed  in  the  speech  acts  and  practices  of  interacting  human  agents. 
The  emphasis  here  is  the  meaning-making  process  rather  than  the  end  product  of  instruction  and  learning. 
Grundy  (1987)  further  contends  that  practical  interest  is  an  interest  in  taking  the  right  practical  action  within 
a  particular  environment  and  is  oriented  toward  the  moral  sphere.  This  is,  indeed,  consonant  with  the 
development  of  cognitive  learning  psychology. 

As  to  a  critical  way  of  knowing,  Grundy  maintains  that  the  emphasis  is  on  critical  reflection  upon 
social  and  cultural  practices.  People  become  conscious  of  the  pre-understandings  in  existing  social  and 
cultural  practices,  uncover  the  contradictions  between  the  ideal  of  truth,  justice,  and  freedom  and  actual 
social  and  cultural  practices,  and  change  social  practices.  Habermas  maintains  that 
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Although  interests  are  fundamental  orientations  of  the  human  species,  they  can 
themselves  be  categorized  either  as  being  stimulated  by  inclination  or  by  principles  of 

reason   It  is  important  to  realize  that  interests  can  also  be  stimulated  by  principles 

of  reason         Habermas  views  persons  as  intrinsically,  or  at  least  potentially,  rational 

beings,  so  interests  which  are  stimulated  by  reason  are  more  fundamental  than  interests 
which  are  stimulated  by  inclination  or  desire  (cited  in  Grundy,  p.17  ). 

Grundy  interprets  Habermas'  fundamental  "pure"  interest  as  being  grounded  in  reason,  an  interest  in 
emancipation.  She  regards  Habermas'  emancipation  as  "independence  from  all  that  is  outside  of  the 
individual  and  is  a  state  of  autonomy  and  responsibility.  More  importantly,  it  is  only  in  the  act  of  self- 
reflection  that  emancipation  is  possible"  (p.  16). 

Streibel  (1991)  has  applied  the  first  two  ways  of  knowing  to  examining  the  practice  of 
instructional  design,  maintaining  that  the  technical  human  interest  approach  to  theory  and  practice  treats 
theory  as  a  guide  to  action.  Traditional  instructional  design  falls  into  this  category.  If  instruction  is 
conceptualized  as  a  set  of  cultural  practices,  a  social  construction  and  meaning-making  process,  it  operates 
at  the  practical  interest  approach.  He  further  states  that  instructional  designers  should  find  ways  to  design 
resources  rather  than  plans  for  teachers  and  learners  if  design  is  regarded  as  process  rather  than  product.  In 
his  words,  "instructional  theories  should  be  treated  as  resources,  rather  than  plans",  because  "all  human 
practice  is  situated  in  an  ongoing  context  that  requires  continual  judgement"  (Streibel,  1991,  p. 851).  As  a 
matter  of  fact,  this  argument  is  similar  to  Winn's  belief  that  the  integrity  of  instruction  lies  in  the  fusion  of 
instructional  design  and  implementation  during  the  real  time  of  teaching.  From  the  various  perspectives  of 
cognitivist  theory,  constructivism,  and  critical  theory,  it  is  apparent  that  why  instructional  theory  should 
serve  as  prescriptive  guidance  for  practitioners,  either  novice  instructional  designers  or  classroom  teachers, 
has  been  a  topic  for  argument  in  the  past  decade.  Are  prescriptive  instructional  theorists  and  those  who 
object  to  the  process  of  prescription  really  focusing  on  the  same  thing,  or  are  they  indeed  talking  about  two 
different  issues?  When  instructional  designers  incorporate  constructivism  into  their  design  of  classroom 
lessons  or  multimedia  learning  environments,  are  they  following  any  prescriptions  or  principles  to  initiate 
their  design  task?  Even  when  designers  are  designing  resources  as  Streibel  recommends,  do  they  not  need 
some  guidance  on  how  to  design  the  kind  of  resources  that  could  enhance  the  meaning-making  process? 

It  seems  that  there  are  two  different  dimensions  of  questions  intertwined  here.  What  are  the 
educational/training  goals  that  orient  instruction  or  instructional  design?  What  kind  of  sources  do 
instructional  designers  draw  upon  to  design  instruction?  Without  keeping  these  questions  clearly  in  mind 
and  dealing  with  them  independently,  the  search  for  a  better  framework  for  designing  instruction  might  be 
futile.  Since  Habermas'  critical  theory  encompasses  all  three  ways  of  knowledge  construction,  we  regard  it 
as  a  valuable  source  to  expand  the  research  and  practice  of  instructional  design. 

From  the  above  theoretical  analysis  of  instructional  design,  it  is  revealing  that  the  development  of 
instructional  design  has  undergone  several  major  theoretical  advances.  The  dimensions  of  instructional 
design  are  broadened  every  few  years,  and  arguments  about  the  theoretical  foundations  for  design  have  never 
ceased.  Rather  than  looking  at  the  evolution  from  behaviorism  to  cognitivism  to  constructivism  as  a 
replacement  process  and  viewing  these  various  paradigms  as  incompatible,  we  view  Habermas'  critical 
theory  as  a  potential  source  for  expanding  our  vision  of  instructional  design. 


ANALYZING  INSTRUCTIONAL  DESIGN  FROM  HABERMAS' 
THEORY  OF  KNOWLEDGE 


In  the  previous  section,  it  has  been  indicated  that  Habermas'  theory  of  knowledge  has  three  forms, 
or  processes,  of  inquiry:  knowledge  that  is  arrived  at  through  empirical-analytic  science,  historical- 
hermeneutical  science,  and  critical  science.  Each  of  the  three  requires  a  different  methodology  for  knowledge 
construction.  According  to  Koetting  and  Januszewski  (1991),  among  the  three  ways  of  deriving  knowledge, 
critical  science  aims  at  deriving  analyses  that  free  consciousness  from  its  dependence  on  hypostatized 
powers.  The  task  of  analyzing  instructional  design  critically  is,  therefore,  targeted  at  enlightening  people's 
many  misconceptions  of  what  instructional  design  should  be.  In  order  to  draw  implications  of  Habermas' 


framework  of  human  interest  on  the  research  and  practice  of  instructional  design,  we  have  organized  his 
framework  into  a  two-dimensional  scheme  by  which  the  various  aspects  of  his  theory  and  its  corresponding 
implications  for  instructional  design  could  be  clearly  presented.  Table  1  summarizes  the  various 
characteristics  of  each  interest. 


Implications  for  Instructional  Design 

The  implications  Habermas'  theory  has  on  instructional  design  also  have  been  organized  around 
several  themes:  (1)  knowledge  construction  process  of  instructional  design,  (2)  the  relationship  between 
theory  and  practice,  and  that  among  learners,  teachers,  designers  and  clients.  Since  this  section  focuses  on 
generating  implications  for  instructional  design,  a  corresponding  table  (Table  2)  of  instructional  design  with 
the  three  interests  is  presented  after  Habermas'  scheme  for  better  comparison. 


Table  1:  Dimensions 

of  Habermas' 

Three  Fundamental  Human 

Interests 

Technical 

Practical 

Emancipatory 

1 

Basic 

Orientation 

controlling 
&  managing 
self,other& 
environment 

understanding 

self,other,& 

environment 

empowering 
those  involved 
in  the 

environment 

2. 

Focus 

product 

process 

praxis 

Goal 

explaining, 
predicting 

meaning- 
making 

justice, 
equality 

On  from  p<i 

correct 
behavior 

rational/ 
moral  action 

responsible, 
autonomous  action 

3. 

Knowledge 
Representation 

facts,  laws, 
procedures 

narrative 
stories 

critical 
theorems 

Knowledge 
Required 

skills 

judgement 

critique 

Knowledge 
Grounded  in 

experience  & 
observation 

understanding 
of  meaning 

critical 
insights 

Knowledge 

Application& 

Construction 

following 
rules 

making 
judgement 

reflecting  upon 
distorted/taken 
for-granted  views 

4. 

Authority 
Resides  in 

plan 

practitioner 

historical 
community 

5. 

Role  of 
Theory 

theories 
guiding 
actions 

theories 
treated  as 
resources 
for  actions 

theorems 
helping 
construct 
new  knowledge 

(Adapted  from  Streibel ,  1 99 1 ) 
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Table  2:  Instructional  Design  with  the  Three  Interests 


Technical 

Practical 

Emancipatory 

1. 

DuMt 

Orientation 

controlling 

Uvll  1 1  v  All  1  It 

&  managingthe 
environment 

understanding 

complexity 

environment 

empowering 
in  a  learning 
environment 

2. 

Focus 

instructional 

nmHiiPt  for 

T's  (L's)  use 

interaction 
nrocess  of 
t/1 

praxis 

Goal 

prespecify 
certain 

Ifjirrii  n  o 

IC-tU  1111 1  iL 

outcomes 

making  the 
learning  & 
teaching 

LwU^  111  1 J 

process 
meaningful 

restructuring 
environments 
that  prohibit 
justice  and 
equality  of  t/1 

Outcomes 

correct, 

r\ rp  c  rv*  c  i  ti  p  f\ 

learning 
outcome 

rational/ 
moral  t/1 
actions 

responsible, 
autonomou 
learning  & 
instruction 

3. 

Knowledge 

Represen- 

*3.tion 

facts,  laws, 
procedures 

narrative 
stories 

critical 
theorems 

Knowledge 
Required 

skills 

judgement 

critique 

K'  rin  \\j  1  pH  o  p 

Grounded 
in 

exnerienoe 

observation 

oft/1 

understanding 
of  meaning 

critical 
insights  into 
t/1 

Application& 
Construction 

tpafhpr*; 
following 
instructional 
rules 

teachers  (L) 
making 
judgement 
on  own  t/1 

teachers  (L) 
reflecting  upon 
distorted/taken 
for-granted  views 

4. 

Authority 
resides  in 

instructional 
plans 

teachers 

teachers' 
community 

5. 

Role  of 
Theory 

instructional 
theories 
guiding  t/1 
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Notions  of  Knowledge 

Technical  interest:  Since  designing  with  the  technical  interest  focuses  on  attaining  prespecified 
outcomes  and  intends  to  derive  a  certain  product,  the  teaching  and  learning  activities  are  often  planned  along 
with  those  outcomes.  In  order  to  plan  these  activities  in  advance,  some  theories  or  principles  are  required 
for  the  designers  to  follow.  Such  theories  or  principles  are  often  generated  by  experiences,  or  observation 
and  synthesis  of  a  number  of  teaching  and  learning  phenomena.  The  result  of  such  design  tends  to  be  a 
prescribed  set  of  instructional  methods  for  teachers'  application.  These  prescribed  methods  may  serve  as  a 
lubricant  to  the  completion  of  a  complicated  design  task  at  best,  and  sacrifice  the  flexibility  of  design  at 
worst.  Hence,  these  methods  have  to  depend  on  an  early  stage  of  formative  evaluation  in  order  not  to 
sacrifice  the  flexibility  and  dynamism  of  learning  and  instruction. 

If  a  prescriptive  theory  is  intended  to  be  applied  by  practitioners,  it  has  to  be  constructed  with 
sensitivity  to  the  ongoing  process  of  teaching/learning,  and  be  embedded  with  formative  research  in  the 
generation  of  theory.  That  is,  formative  research  should  be  an  integral  part  of  a  prescriptive  theory 
(Reigeluth,  1993),  and  a  prescriptive  theory  can  be  viewed  simply  as  a  blueprint  or  prototype  for  another 
better  prescriptive  theory,  which  will  not  be  completed  until  it  incorporates  the  needs  of  users  within  a 
specific  context.  While  the  needs  and  characteristics  of  users  and  the  conditions  of  a  context  or  environment 
are  assessed  before  instruction,  it  is  also  important  to  continuously  diagnose  the  emergent  needs  in  the 
course  of  real  teaching-learning  interaction. 

Practical  interest:  Therefore,  when  a  prescriptive  theory  is  formed  in  such  a  manner,  it  has  more  a 
spirit  of  the  practical  interest  than  of  the  technical  interest.  However,  the  outcomes  and  conditions  of 
instruction  have  to  be  defined  and  redefined  during  interaction,  not  simply  interpreted  beforehand. 
Otherwise,  it  will  still  be  designing  with  the  technical  spirit  disguised  by  a  practical  facade. 

As  a  result,  such  a  move  toward  the  practical  interest  will  encourage  practitioners  to  make  their 
own  judgements  on  what  is  appropriate  for  themselves.  An  instructional  theory  or  a  set  of  principles  is 
hence  regarded  as  a  resource  for  the  teachers  to  adapt  and  modify  their  instruction  based  on  their  judgement. 
Designers  along  with  teachers  in  this  circumstance  will  use  their  understanding  of  the  processes  of  teaching 
and  learning  to  construct  their  knowledge  about  learning  and  instruction,  which  will  better  improve  their 
conception  and  practice  of  design.  What  can  best  capture  such  a  process  might  be  narrative  stories  that 
present  the  richness  of  the  meaning-making  process  of  teaching  and  learning,  since  they  can  inspire  the 
practitioners  to  make  their  own  judgements. 

Emancipatory  interest:  The  emancipatory  interest  to  designing  instruction  sees  the  beliefs  and 
ideologies  that  designers  hold  as  a  potential  impediment  against  the  empowerment  of  those  involved. 
Designers  who  espouse  the  emancipatory  interest  hence  look  into  the  ideologies  dominating  themselves  and 
others,  and  examine  how  these  might  prohibit  them  from  carrying  out  their  understanding  into  actions. 
When  design  is  penetrated  by  the  ideologies  which  are  identified  as  viable  and  applicable  without  taking  into 
account  whose  interests  are  served,  genuine  justice  and  equality  will  not  be  obtained.  Indeed,  the  extent  to 
which  certain  ideologies  are  recognized  and  accepted  often  determines  the  degree  to  which  the  existing  power 
relationship  and  existing  interests  will  be  maintained.  But  how  could  designers  capture  their  own  distorted 
beliefs  and  values,  and  in  the  meantime,  empower  those  involved  to  look  into  theirs  in  order  not  to  be 
confined  within  those  social  and  cultural  constraints? 

According  to  Grundy  (1987),  "one  of  the  fundamental  ways  in  which  ideological  oppression 
operates  is  to  make  that  which  is  cultural  (and  hence  in  principle  susceptible  to  change)  appear  natural  (and 
hence  unchangeable)"  (p.  104).  However,  since  ideologies,  as  Grundy  contends,  are  the  dominant  ideas  and 
thinking  of  a  group  or  culture  it  is  often  overlooked  that  our  common-sense  understandings  of  reality  may 
involve  some  unrecognized  ionns  of  domination.  Emancipation  from  the  predominant  ideologies  may  not 
be  possible  simply  through  processes  of  reflection,  such  as  those  which  characterize  the  practical  interest. 
Emancipation  should  be  built  upon  continuous  reflection  upon  these  beliefs  and  ideologies. 


It  is  not  until  design  is  aimed  toward  uncovering  underlying  beliefs  and  values  that  we  can  better 
rise  up  from  the  confinement  of  ideological  oppression.  Therefore,  the  emancipatory  design  approach 
constructs  knowledge  through  continuous  problematization:  "recognizing  the  problematic  nature  of 
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existence  is  [becomes]  essential  in  order  to  address  questions  about  the  root  causes  of  problematic  aspects  of 
life  and  address  possibilities  of  change"  (Grundy,  1987,  p.  104).  The  design  efforts  of  the  emancipatory 
interest  are  not  expended  on  constructing  instructional  methods  for  teachers  to  apply  nor  on  merely 
encouraging  practitioners'  judgements  about  their  own  teaching  and  learning.  Rather,  they  will  aim  at 
liberating  the  practitioners  from  the  confinement  of  any  social  or  cultural  taken -for-granted  views,  focusing 
on  what  could  be  reconstructed,  and  enabling  them  to  take  responsible  and  autonomous  actions. 

Since  the  knowledge  construction  process  of  emancipatory  design  is  engaged  by  not  only  designers 
but  also  teachers  and  learners,  designers  could  better  reflect  upon  the  distorted  beliefs  through  critical 
theorems. 

Grundy  explains  that 

a  critical  theorem  is  a  theoretical  reconstruction  of  the  undistorted  human  competencies 
through  which  the  human  species  has  constituted  itself;  it  is  a  theory  about  fundamental 
human  capacities,  undistorted  by  the  operation  of  ideology,  which  have  been  the  basis  for 
the  species'  evolution.  These  critical  theorems  are  implicit  in  the  very  nature  of  human 
interaction  and  thus  represent  a  potential  for  enlightenment  and  emancipation  (1987, 
pp. 111-112). 

These  critical  theorems  are  also  used  to  assist  the  teachers  and  learners  to  penetrate  their  own 
confined  views,  and  reconstruct  new  knowledge  for  learning  and  teaching.  Nevertheless,  the  creation  of 
critical  theorems  is  not  the  ultimate  goal  of  design  for  those  who  hold  the  emancipatory  interest. 
Following  this  would  be  the  organization  of  processes  of  enlightenment  and  the  conduct  of  the  political 
struggle,  both  of  which  were  suggested  by  Habermas.  But  how  can  the  designer  help  teachers  and  learners 
become  enlightened  on  the  problematic  aspects  of  their  learning  and  teaching? 

Design  Incorporates  Processes  of  Enlightenment 

The  design  knowledge  that  is  constructed  in  the  emancipatory  mode  has  to  be  authentic  from  the 
teachers'  and  learners'  perspectives,  not  the  designers',  since  it  is  they  who  make  sense  of  whether  it  is 
authentic  or  not.  Such  authentic  knowledge  involves  practitioners  in  authentic  experiences  through  critical 
self-reflection  upon  personal  practice.  McTaggart  and  Singh  (1986)  claim  that 

"critical  reflection  involves  more  than  knowledge  of  one's  values  and  understanding  of 
one's  practice.  It  involves  a  dialectical  criticism  of  one's  own  values  in  a  social  and 
historical  context  in  which  the  values  of  others  are  also  crucial.  Criticism  itself  is, 
therefore,  a  relational  concept;  criticism  can  only  be  conducted  in  a  community  where 
there  is  determination  to  learn  rationally  from  each  other"  (p.43). 

Since  authentic  knowledge  goes  hand  in  hand  with  critical  self-reflection,  the  two  will  orient  the  teacher  and 
learner  toward  the  processes  of  enlightenment  of  their  own  problems.  The  constraints  that  are  culturally 
shaped  (and  hence  are  susceptible  to  change)  will  be  the  targets  for  critical  reflection  in  the  design  process. 
Designers,  who  focus  on  distinguishing  natural  from  cultural  constraints,  will  hence  engage  themselves  and 
all  others  in  examining  the  problematic  aspects  of  cultural  constrains.  The  destiny  design  leads  to  is  the 
uncovering  of  factors  that  prohibit  and  prevent  the  autonomous  actions  of  learning,  instruction  and  design 
respectively.  That  is  to  say,  the  enlightenment  processes  are  mere  concerned  with  the  clarification  of  root 
problems  of  learning,  teaching  and  design  than  with  what  is  to  be  and  how  it  should  be  learned,  taught  and 
designed.  It  is  a  fundamental  approach  to  enabling  practitioners  to  analyze  and  diagnose  their  own 
learning/teaching  problems,  and  to  reflect  upon  why  they  teach  or  learn  the  way  they  do  and  what  they 
might  reconstruct. 

In  many  cases,  instructional  designers  do  emphasize  to  a  large  extent  the  identification  of 
problems.  However,  such  problem  identification  is  often  confined  within  certain  parameters.  It  is  within 
the  boundary  of  such  parameters  that  designers  can  not  free  themselves  from  the  historical  or  social 
constraints  on  their  practice.  There  are  some  cases  in  which  instructional  designers  do  capture  the  structural 
tyranny,  and  realize  that  fundamental  change  has  to  be  enacted  from  the  organizational  or  institutional  level. 
But  even  so,  it  is  very  rare  that  teachers  or  learners  are  involved  altogether  in  examining  how  their  beliefs 
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and  values  have  been  shaped  by  institutions,  and  how  the  institutions  in  turn  have  shaped  their  way  of 
thinking,  believing  and  acting.  Therefore,  even  when  instructional  designers  attempt  to  identify,  analyze 
and  solve  problems,  it  does  not  necessarily  mean  they  are  engaging  in  critical  awareness  of  their 
environmental  and  cultural  constraints. 

Cosgrove  (1982)  contends  that  meaningful  learning  cannot  be  separated  from  autonomous  learning. 
In  order  to  make  learning  experiences  themselves  meaningful,  the  conditions  under  which  learning  occurs 
must  be  fundamentally  changed.  This  idea  has  profound  implications  for  design.  It  is  important  that 
instructional  designers  should  not  be  satisfied  with  merely  identifying  the  existing  conditions  and 
constraints,  but  they  should  take  a  further  step  to  reconstruct  the  constraints,  and  to  provide  conditions 
under  which  learning  will  best  occur.  To  put  it  differently,  designers  must  improve  their  understanding  of 
teaching/learning  theories;  they  must  also  understand  the  context  of  learning/teaching,  being  ready  to 
challenge  that  context  if  it  prohibits  the  teacher  and  learners  from  becoming  autonomous.  Designers  should 
also  continuously  remind  themselves  that,  while  acting  collaboratively,  they  may  not  engage  in  reflective 
action  at  all;  the  two  are  not  equivalent.  To  adopt  the  emancipatory  approach  to  design,  any  design  action 
must  be  reflectively  generated.  The  purpose  of  taking  responsible  and  autonomous  actions  is  to  transcend 
the  barriers  to  such  reflection.  In  one  sentence,  the  enlightenment  process,  as  a  major  characteristic  of 
emancipatory  design,  is  a  continuous  problematizing  process. 

Relationship  between  Theory  and  Practice 

Based  on  the  above  clarifications,  we  could  draw  a  conclusion  that  the  various  approaches  to  design 
imply  different  relationships  between  theory  and  practice. 

Technical  interest:  An  instructional  theory  embodying  the  technical  interest  is  applied  as  if  it 
were  a  natural  science.  "It  is  assumed  in  order  to  guide  practice  and  to  test  a  sounding  theory"  (Streibel, 
1991,  p.860).  Such  design  efforts  lean  toward  the  side  of  theory,  which  is  regarded  as  a  higher-level  of 
knowledge.  It  is  revealing  that  much  of  the  practice  of  instructional  design  falls  into  this  category.  In 
many  cases,  skills  acquisition,  rather  than  the  process  of  learning  and  teaching,  has  been  regarded  as  the  top 
priority.  Many  instructional  models  also  focus  on  the  specification  of  types  of  knowledge,  and  optimal 
ways  for  teaching.  Gagne's  conditions  of  learning  and  nine  events  of  teaching  is  a  good  example  of 
designing  learning  and  instruction  with  the  technical  interest  in  mind.  If  an  instructional  designer  simply 
follows  his  theory  to  approach  a  design  task,  even  though  he  might  accomplish  an  efficient  task,  he  is 
simply  practicing  design  technically. 

Practical  Interest:  However,  meaning  is  socially  and  culturally  constructed  through  human 
interaction,  as  claimed  in  the  beginning  of  this  section.  The  practical  interest  is  a  fundamental  interest  in 
understanding  the  environment  through  interaction  based  on  a  consensual  interpretation  of  meaning. 
Therefore,  a  theory  in  the  practical  interest,  from  Grundy's  view,  is  "judged  according  to  whether  the 
interpreted  meaning  assists  the  process  of  making  judgements  about  how  to  act  rationally  and  morally  in 
the  world"  (p.  14).  With  such  an  orientation  of  design,  theory  and  practice  inform  each  other,  and  more 
important,  theory  is  constructed  through  the  meaning-making  processes  engaged  most  of  the  time  by 
learners  and  teachers,  unless  designers  themselves  play  participant  observers  at  a  deep  and  persistent  level. 

Emancipatory  interest:  An  instructional  theory  with  the  emancipatory  interest  will  manifest 
the  ideas  of  truth,  justice  and  freedom,  which  are  regarded  as  "transcendental  realities  within  everyday  human 
interaction  rather  than  within  some  external  reality"  (Habermas,  1972,  cited  in  Streibel,  1991,  p.858). 
Since  emancipation  is  a  potential  waiting  to  be  realized  in  the  world  of  every  human  being,  designers  have 
to  first  become  critically  conscious  and  aware  of  how  they  construct  their  current  knowledge  in  order  to 
realize  emancipation.  They  also  need  to  socially  reconstruct  their  knowledge,  beliefs  and  practices,  critically 
reflect  on  social  and  cultural  practices,  and  finally  restructure  their  future  actions.  The  relationship  between 
an  instructional  theory  and  practice  within  emancipatory  design,  therefore,  contains  political  elements 
through  which  teachers'  and  learners'  autonomy  and  responsibility  can  be  promoted  through  collective  and 
reflective  efforts.  The  political  struggle  here  lies  in  the  fact  that  the  theory  focuses  on  consciousness  of  the 
pre-understandings  in  existing  social  and  cultural  practices.  The  struggle  also  attempts  to  reveal  the 
contradictions  between  the  ideals  of  truth,  justice  and  freedom,  and  actual  social  and  cultural  practices. 


More  important,  the  theory  does  not  simply  guide  teachers'  and  learners'  action;  neither  does  it 
assist  their  judgement-making  on  their  own  practice.  Rather,  it  helps  designer,  teachers  and  students 
struggle  to  change  their  social,  economic  and  cultural  conditions.  Such  an  approach  to  design  will 
ultimately  lead  to  restructuring  the  whole  learning  and  teaching  system,  and  the  efforts  that  more  and  more 
instructional  theorists  are  making  in  redesigning  the  educational  system  make  this  possibility  viable.  The 
emancipatory  design  obviously  takes  into  account  more  fundamental  issues  about  learning  and  teaching. 

Implications  on  the  Relationships  between  Designers,  Teachers.  Learners  and  Clients 

The  relationship  between  theory  and  practice  also  underlies  the  power  relationships  between 
designer's,  learners,  teachers  and  clients  embedded  in  the  design  process.  Designers  should  recognize  that  the 
traditions  of  the  division  of  responsibility  and  the  distribution  of  power  in  the  work  of  design  are  strongly 
embedded  in  the  histories  of  designers,  their  clients,  teachers  and  learners.  To  examine  the  existing  power 
relationships  in  a  design  task,  designers  need  to  locate  themselves  inside  the  community  of  instructional 
designers,  teachers,  learners  and  any  other  groups  which  are  involved.  This  is  Habermas'  (1972)  belief  that 
the  authority  of  the  emancipatory  interest  resides  in  the  historical  community,  not  simply  a  particular  group 
regarded  in  isolation  from  its  tradition;  the  social,  cultural  and  historical  factors  are  the  essence  of  such  a 
community.  After  positioning  themselves  within  the  realm  of  such  a  historical  community,  designers  will 
be  able  to  see  more  clearly  how  their  beliefs  and  values  are  shaped;  by  doing  so  designers  can  then  transcend 
the  limitations  of  their  current  practice  and  broaden  their  visions  to  other  design  possibilities. 

Especially  for  those  who  endeavor  to  restructure  the  educational  system  to  foster  learners' 
independent  and  autonomous  learning,  the  recognition  of  historical  community  is  important.  Therefore, 
within  the  emancipatory  design  framework,  the  designer  is  no  longer  the  one  who  designs  instruction  but 
the  one  who  is  educated  in  the  learning/  teaching  processes.  The  learners  and  teachers,  in  turn,  while  being 
the  targets  for  liberation,  also  liberate  designers  by  presenting  designers  with  new  insights  into  their  design 
practice.  In  addition,  the  teacher,  as  stated  by  Freire  (1972),  is  "no  longer  the-one-who-teaches,  but  one 
who  in  himself  taught  in  dialogue  with  the  students,  who  in  their  turn,  while  being  taught,  also  teach" 
(p.53).  Freire  does  not  mean  that  the  teacher  no  longer  has  any  role  in  the  selection  of  knowledge  for  study, 
for  the  character  of  liberating  education  is  dialogical  not  monological.  This  means  that  [the  designer, 
teacher  and  students]  all  have  the  right  and  responsibility  for  contributing  to  the  process  of  [learning, 
teaching  and  design]  (Freire,  1972,  p.92). 

The  instructional  designer  must  assume  a  new  role  within  this  emancipatory  framework.  When 
design  efforts  are  oriented  toward  the  emancipatory  interest,  an  instructional  consultant  rather  than  an 
instructional  designer  will  be  more  appropriate  for  the  task.  Using  Habermas'  (1972)  analogy,  the 
instructional  consultant  will  act  like  a  psychoanalyst,  whose  dialogue  with  his/her  client  focuses  on  self- 
enlightenment  and  problem-solving.  It  can  be  said  that  the  instructional  consultant  will  not  only  think  and 
act  in  a  dialogical  and  reflected  mode,  but  also  negotiate  with  any  party  line  which  may  prohibit  the 
reconstruction  of  cultural  and  social  constraints  on  the  processes  of  learning  and  teaching.  The  ethical  and 
moral  responsibility  is  a  new  dimension  for  consideration.  Similar  to  those  who  engage  in  educational 
systems  redesign,  efforts  are  expected  to  be  expended  on  overcoming  the  political,  social,  cultural  and 
environmental  barriers  embedded  within  the  system. 

From  the  above  description,  it  is  obvious  that  much  practice  of  instructional  design  still  embodies 
many  characteristics  of  a  technical  nature.  Many  people  have  now  started  to  design  their  instruction  by 
focusing  on  the  resources  or  learning  environment,  which  can  provide  learners  with  a  greater  space  for 
navigating  and  inquiring.  Applying  constructivism  and  situated  learning  to  instructional  design  is  no 
longer  a  dream.  Vanderbilt  group's  anchored  instruction  and  Duffy  and  Jonassan's  (1993)  design  of  a 
constructivist  learning  environment  both  demonstrate  such  a  possibility.  Another  example  is  Tripp  and 
Bichelmeyer's  (1991)  rapid  prototype,  in  which  learning/instructional  goals  and  methods  are  selected  based 
on  the  continuous  interaction  among  the  designer,  teacher,  learners  and  the  client.  However,  while  there  are 
more  and  more  design  issues  brought  up  to  accommodate  the  practical  interest,  interest  in  the  emancipatory 
design  has  not  yet  been  widespread.  Moreover,  these  design  models  or  approaches  embody  elements  of 
more  than  one  interest.  Our  contention  is  that  Habermas'  three  knowledge  interests  are  not  supposed  to 
dominate  our  thinking  and  action  in  aclear-cut'manner.  Although  one  particular  interest  may  dominate  our 
decision- making  process  and  actions,  we  do  not  have  to  be  restricted  to  it  if  the  other 
two  have  value. 
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ASSUMPTIONS  FOR  APPLYING  THE  THREE 
KNOWLEDGE  INTERESTS  TO  DESIGN 


How  Do  the  Various  interests  Orient  Our  Actions? 

The  first  question  we  need  to  ask  is,  do  Habermas'  three  knowledge  interests  always  operate  in 
insolation  or  interact  to  some  extent  within  our  cognition  and  action?  Although  it  appears  that  each  of  the 
three  fundamental  human  interests  has  a  distinct  set  of  characteristics,  we  need  to  inquire,  in  reality,  whether 
our  cognition  and  actions  fit  neatly  into  simply  one  interest  or  another. 

Living  in  this  postmodern  society,  pluralism  and  multiplicity  have  received  gTeat  attention  and  are 
widely  discussed  in  education.  People  always  have  different  intentions,  expectations  and  attitudes  when 
approaching  a  task.  Even  for  the  same  person,  he/she  might  have  various  intentions  and  expectations  of, 
and  attitudes  toward,  various  tasks,  at  different  stages  of  his/her  life.  Although  Habermas  (1972)  argues 
that  the  emancipatory  interest  is  the  highest  form  of  human  rationality,  it  is  not,  in  fact,  the  only  interest 
dominating  every  action  we  take.  Our  lives  are  multi-dimensional,  and  hence  at  different  times  and  in 
different  situations  we  often  take  different  actions. 

Design  tasks  are  human  activities  complicated  by  many  social,  cultural,  and  even  political  and 
economic  interplays.  A  design  task  often  involves  various  domains  of  problems  to  be  solved,  which 
requires  different  types  of  knowledge  and  actions.  We  may  have  an  ultimate  goal  guiding  our  action,  but  in 
order  to  achieve  that  goal,  we  might  need  to  undertake  various  kinds  of  efforts.  Design  is  usually  conducted 
and  carried  out  through  diverse  considerations:  a  designer  often  needs  to  negotiate  among  different 
stakeholders.  Toward  which  of  the  three  interests  a  design  task  is  oriented  is  thus  influenced  by  these 
considerations. 


What  Are  the  ^Relationships  among  the  Three  Interests? 

When  looking  into  the  relationship  among  the  three  interests,  Grundy  (1987)  maintains  that  while 
the  practical  interest  is  compatible  with  the  emancipatory  interest,  the  technical  interest  is  not.  Does  this 
mean  that  the  practical  and  emancipatory  interests  might  orient  one's  cognition  and  action  simultaneously, 
but  not  the  emancipatory  and  the  technical?  Or  could  the  technical  interest  operate  at  a  different  level  from 
the  practical  and  the  emancipatory?  If  only  two  of  them  can  coexist,  does  it  preclude  the  application  of  the 
third  one? 

Theoretically,  the  three  interests  are  different  from  one  another  in  many  domains.  But  realistically, 
each  of  the  interests  exerts  different  degrees  of  influence  on  people's  thinking  and  action  when  they  engage 
in  a  task.  After  all,  is  the  technical  interest  a  preparation  for,  or  inhibition  of,  the  practical  interest?  Will 
the  practical  interest  naturally  lead  to  the  emancipatory  interest  because  of  their  compatibility?  Do  people 
only  need  to  adopt  one  interest  to  solve  different  problems?  Or  do  the  three  interests  indeed  operate  at 
various  levels? 

Is  Any  of  the  Three  Interests  Worse  than  Another? 

In  answering  the  above  questions,  we  have  to  ask  a  relevant  question:  Is  any  one  of  the  three 
interests  inherently  worse  than  another  when  it  dominates  the  practice  of  instructional  design?  Or  is 
emancipatory  design  always  the  best?  If  we  think  the  practical  interest  is  better  than  the  technical,  then  we 
should  direct  our  efforts  toward  the  practical  mode  of  thinking  and  action.  Similarly,  if  we  agree  that  the 
emancipatory  interest  is  the  highest  form  of  human  rationality,  as  justified  by  Habermas  (1972),  and  believe 
that  it  should  always  direct  our  actions,  then  we  certainly  should  eliminate  any  possible  actions  that  serve 
either  the  technical  or  practical  interest.  Habermas  seems  to  value  the  emancipatory  interest  more  than  the 
other  two  when  maintaining  that  it  has  the  purest  rationality.  However,  when  we  apply  it  to  the  analysis 
of  instructional  design,  does  it  mean  we  should  completely  abandon  the  kind  of  design  practice  that  is 
dominated  by  the  technical  or  practical  interests? 


The  Significance  of  Taking  into  Account  All  Three  t 


The  significance  of  taking  all  three  interests  into  account  while  engaging  in  a  design  task  is  that 
we  need  more  alternatives  to  broaden  our  vision  of  design,  and  expand  our  considerations  about  design  in  the 
complex  human  arena.  If  we  limit  ourselves  to  only  one  single  interest,  we  will  lose  sight  of  other 
possibilities.  The  critical  point  is  our  ability  to  determine  which  mode  of  actions  to  take  when  coping  with 
a  certain  problem.  It  is  also  essential  to  identify  the  conducive  situations  for  each  of  the  three  interests  to 
play.  More  important,  the  kind  of  efforts  required  by  each  interest  should  be  recognized. 

In  the  evolution  of  any  discipline,  we  could  find  that  new  ways  of  thinking  and  doing  always  gain 
popularity  when  they  are  more  accountable  to  some  phenomena  than  the  old  ones.  Nevertheless,  even 
though  our  knowledge  keeps  advancing,  there  remain  some  aspects  of  human  activities  which  could  still  be 
better  done  in  the  old  ways.  While  we  should  not  let  the  old  paradigms  of  thinking  prevent  our  progress, 
we  may  not  make  a  wise  decision  when  trying  to  replace  everything  with  a  new  paradigm.  The  purpose  of 
looking  at  the  values,  weaknesses  and  strengths  of  each  of  the  three  interests  in  determining  designing 
efforts  is  to  incorporate  as  many  design  possibilities  as  is  useful.  We  need  to  have  a  larger  repertoire  to 
accommodate  the  needs  of  a  wider  range  of  contexts.  Moreover,  the  routes  to  the  highest  goal  are  many;  we 
need  to  identify  as  many  of  them  as  possible,  rather  than  choosing  only  one  route. 


NEW  PERSPECTIVES  ON  INSTRUCTIONAL  DESIGN 


In  the  previous  sections,  we  have  provided  critical  perspectives  on  the  practice  and  research  of 
instructional  design.  We  first  have  summarized  the  various  dimensions  of  Habermas'  three  fundamental 
human  interests  into  a  table  so  as  to  draw  implications  for  instructional  design.  However,  we  have  only 
focused  on  the  notions  of  knowledge,  the  relationship  between  theory  and  practice  and  the  relationships 
between  designers,  teachers  and  clients.  The  main  purpose  of  such  a  critical  analysis  has  been  to  inquire 
how  the  three  interests  might  be  applied  to  the  design  of  learning  and  instruction,  and  why  it  is  essential  to 
take  all  three  interests  into  account.  More  important,  the  analysis  serves  to  generate  a  set  of  new 
perspectives  for  instructional  design. 


Oneness  of  Learnin2^Irislruction  and  Design 

Traditionally,  learning  theories,  instructional  theories  and  instructional  design  theories  are  treated 
as  three  distinct  entities.  While  the  numerous  arguments  about  whether  instructional  theories  can  stand 
alone  from  learning  theories  are  made,  recent  exploration  of  cognitive  psychology  provides  a  great  deal  of 
insights  into  the  possibilities  of  integrating  the  two.  However,  beyond  the  integration  of  learning  and 
instruction  there  is  a  certain  point  where  learning,  teaching  and  design  can  come  together.  That  is  when 
learners,  instructors,  and  designers  engage  all  together  in  reflective  and  critical  thinking  to  explore  the 
fundamental  problems  from  diverse  perspectives.  It  is  our  belief  that  instructional  designers  or  teachers 
should  be  particularly  reflective  and  critical  in  thinking  and  dialectic  in  interacting,  and  respect  a  world  of 
plurality  instead  of  certainty  and  control,  so  that  a  journey  of  instructional  design  will  be  redirected  to 
unlimited  possibilities. 


Transactional  Roles  of  Learners.  Teachers  and  Designers 

Instructional  designers  who  have  taken  for  granted  that  their  knowledge  of  instruction  exceeds  that 
of  many  teachers  need  to  reflect  on  the  complex  interactions  between  learning  and  instruction.  Learners 
should  be  given  the  responsibility  to  direct  and  manage  their  own  learning.  When  they  become  self- 
teachers,  they  will  be  more  creative.  In  addition,  teachers'  ongoing  self-awareness  and  self-reflection  will 
enable  them  to  perceive  instruction  in  adynamic  way;  teachers'  and  learners'  reflection  will  allow  them  to 
make  decisions  about  the  kinds  of  instruction  and  learning  appropriate  for  learners.  The  mission  of 
instructional  designers  is  to  enable  teachers  to  reflectively  design  instruction  that  can  empower  students  to 
design  learning  for  themselves. 


In  a  learning-centered  environment,  instructional  designers  will  be  the  backstage  heroes,  assisting 
learners  and  teachers  to  become  the  main  actors  of  the  learning  drama  rather  than  controlling  the  learning 
environment.  All  learners,  teachers  and  designers  should  engage  in  the 

process  of  learning  and  deepen  their  understanding  of  the  purpose  of  teaching  and  design.  Knowledge  about 
learning  theories  or  instructional  theories,  or  even  design  theories  is  not  the  privilege  of  any  one  of  these 
groups,  although  the  extent  to  which  they  need  to  know  these  varies  from  one  group  to  another.  Designers 
should  not  limit  themselves  to  the  task  of  design.  With  a  deep  understanding  of  the  complexity  of  teaching 
and  learning,  and  insights  into  human  nature  and  potential,  designers  will  no  longer  confine  themselves 
within  a  certain  paradigm  or  approach,  but  continuously  explore  new  possibilities.  In  summary,  the  new 
role  of  the  instructional  designer  is  transactional  with  that  of  learners,  teachers  and  client  in  both  the  local 
and  general  contexts. 


fJew  Perceptions  of  Designers 

Instructional  designers  need  to  recognize  the  uncertainty  in  any  design  context,  since  reality  is 
constantly  changing.  A.  mindset  that  can  accommodate  such  uncertainty  must  be  cultivated.  They  should 
have  the  ability  to  grasp  the  wholeness  and  originality  of  the  phenomena  they  observe  without  projecting 
their  own  personal  biases.  By  doing  so,  they  can  see  and  detect  the  hindrances  in  learning  and  teaching,  and 
hence  find  an  appropriate  way  to  loosen  them.  They  also  should  make  keen  observation  of  the  learning 
context  so  that  they  can  adapt  their  design  whenever  it  is  necessary.  Their  readiness  for  adapting  design 
approaches  is  very  important.  'With  these  capabilities,  instructional  designers  can  play  several  different 
roles. 

The  roles  an  instructional  designer  can  play  are  many.  In  the  critical  analysis  section,  we  have 
pointed  out  that  when  an  instructional  designer  practices  the  emancipatory  approach,  he/she  is  no  longer  an 
instructional  designer  but  a  consultant  who  is  concerned  with  more  than  the  content  of  learning  and 
teaching.  The  consultant  is  like  the  psychoanalyst  who  helps  his/her  clients  to  discover  their  own 
problems  and  assumes  responsibility  to  solve  them.  He/she  will  focus  on  ways  of  enlightening 
teachers/clients. 

Indeed,  if  an  instructional  consultant  has  a  philosopher's  curiosity  about  what  life  really  is,  a 
psychologist's  insights  into  human  emotions  and  personalities,  and  a  sociologist's  concern  with  the 
complex  relationships  entangled  in  our  reality,  he/she  will  expand  his/her  own  repertoire  for  design. 
Designers  need  to  learn  to  think  like  historians,  anthropologists,  inventors,  and  even  artists.  This  does  not 
mean  designers  have  to  master  the  knowledge  of  every  discipline.  What  is  proposed  here  is  a  new 
perception  and  expectation  of,  and  new  attitude  toward,  what  we  have  being  calling  "instructional 
designers". 

As  argued  earlier,  skills  for  design  are  important,  but  critical  insights  are  what  advance  the 
discipline  of  instructional  design.  Designers  should  not  only  focus  their  attention  on  the  processes  of 
learning  and  teaching,  but  on  the  social,  political  and  economic  impact  on  these  processes.  Even  the  most 
advocated  view  among  instructional  designers,  a  systemic  approach  to  design,  requires  designers  to 
continuously  reexamine  their  relationships  with  others  who  are  also  involved  in  learning  and  teaching. 
Within  the  larger  system,  the  instructional  consultant  should  not  only  think  and  act  in  a  dialogical  and 
reflective  mode,  but  also  negotiate  with  any  party  line  which  may  prohibit  the  reconstruction  of  cultural  and 
social  constraints  on  the  processes  of  learning  and  teaching.  This  ethical  and  moral  responsibility  presents 
a  new  dimension  for  consideration,  and  efforts  should  be  made  to  overcome  the  political,  social,  cultural  and 
environmental  barriers  embedded  within  the  system. 


Djrect  Pointin.fi  to  the  Problems;    from  Problem-Solving  _  Design  to  Enablioe  Pegfen 

When  an  instructional  consultant  assumes  more  social  responsibilities,  the  ultimate  goal  of 
instructional  consultation  is  no  longer  solving  some  particular  teaching  or  learning  problems.  It  is  to 
enable  learners  or  teachers  to  diagnose  problems  for  themselves,  and  search  for  solutions  to  these  problems. 
More  fundamentally,  the  consulting  process  is  a  continuous  and  ongoing  problematizing  process.  The  root 


causes  of  the  problems  are  traced  not  only  to  actions,  but  also  to  one's  ideology  and  beliefs.  The  purpose  is 
to  uncover  factors  that  prohibit  and  prevent  the  autonomous  actions  of  learning,  instruction  and  design, 
respectively.  It  is  a  fundamental  approach  to  enabling  practitioners  to  analyze  and  diagnose  their  own 
learning/teaching  problems,  and  to  reflect  upon  why  they  teach  or  learn  the  way  they  do  and  what  they 
might  reconstruct. 

Much  of  the  current  emphasis  on  problem  analysis  in  the  process  of  design  still  operates  at  a 
superficial  level.  It  is  often  confined  within  certain  parameters,  within  which  designers  could  not  free 
themselves  from  the  historical  or  social  constraints  on  their  practice.  Especially  when  politics  is  involved, 
proposed  solutions  to  the  problems  often  end  up  conforming  to  those  who  have  the  power  to  make 
decisions.  Therefore,  the  move  from  a  problem-solving  design  to  an  enabling  or  empowering  design 
requires  us  to  look  into  tne  depths  of  a  problem  in  order  to  find  the  fundamental  causes  prohibiting  one's 
ability  to  undertake  the  problem  independently.  Such  a  role  for  the  instructional  consultant  will  demand 
more  discipline. 


Travel  Acruss  the  Boundaries 

An  instructional  consultant  should  be  able  to  travel  across  the  boundaries  of  the  three  approaches 
to  design.  The  three  approaches  suggested  in  this  study  could  provide  us  with  more  design  or  consultation 
considerations.  While  we  should  not  resist  the  guidance  of  theories  or  principles,  we  should  always  look 
into  the  limitations  of  each  of  the  approaches.  Some  of  their  limitations  may  be  inherent,  embedded  in  the 
underlying  assumptions  of  the  approach.  However,  many  more  limitations  are  imposed  by  those  who 
undertake  them  if  they  are  not  able  to  flexibly  apply  these  approaches,  or  elevate  themselves  beyond  the 
boundary  of  a  particular  approach. 

But  to  be  able  to  freely  and  creatively  use  the  three  approaches,  an  instructional  consultant  would 
regard  skills  or  methods  as  merely  stepping  stones  for  learning  and  teaching.  Skills  which  are  identified  as 
°ood  and  useful  will  be  changed,  and  even  discarded  at  certain  points  in  time,  so  that  new  skills  can  be 
constructed.  The  purpose  is  not  to  demean  skills,  but  to  point  out  the  limitations  of  ,  lem,  since  too  much 
emphasis  on  skills  tends  to  prevent  learners  and  teachers  from  searching  for  other  possibilities.  Learning 
can  be  constructed,  deconstructed  and  reconstructed  again.  There  are  no  fixed  rules  for  learning  or  teaching. 
Without  such  an  understanding,  it  will  be  approached  always  within  the  skill  paradigm. 

Instructional  design,  in  the  new  light  through  Habermas'  knowledge  framework,  demands  the 
cultivation  of  visions  and  perspectives,  and  requires  humanistic  concerns  and  more  discipline.  In  addition  to 
familiarity  with  the  three  design  approaches  (technical,  practical  and  emancipatory),  practitioners  who  wish 
to  design  practically  and  critically  have  to  possess  the  capability  to  grasp  the  situational  and  dynamic  nature 
of  learning  and  teaching.  Those  who  have  practiced  design  for  a  long  time  may  not  necessarily  be  better 
than  the  novice  in  applying  design  if  they  are  confined  within  their  long-held  design  beliefs,  resisting  new 
possibilities  and  new  visions  of  design.  They  should  also  learn  to  see  a  design  task  holistically  and 
situationally,  without  segregating  learners  from  teachers  and  designers,  or  even  the  process  of  learning  from 
teaching  and  design.  Considerations  of  design  are  based  on  broader  and  deeper  foundations.  Methods  of 
learning,  teaching  and  design  are  simply  tools  to  reach  the  ultimate  goal;  they  are  means,  not  ends,  of 
design  efforts.  Therefore,  design  should  not  end  with  the  construction  of  methods  for  application.  In  the 
process  of  instructional  design,  designers  should  committ  themselves  to  transforming  reality  when 
necessary,  not  simply  to  designing  a  learning/teaching  task.  However,  we  should  recognize  that  methods 
are  vitalized  by  those  who  use  them,  not  the  other  way  around.  Methods  themselves  have  no  life;  it  is  the 
person  who  knows  when  and  how  to  use  which  methods  so  as  to  make  a  certain  approach  more  appealing 
than  others. 
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Introduction 


In  recent  years,  the  development  of  computer  based  learning  environments  has  been  influenced  by  the 
development  of  more  powerful  computer  technologies  and  theoretical  orientations  .  On  the  technology  side, 
we  have  more  and  more  powerful  systems  which  are  capable  of  delivering  sophisticated  multimedia 
(computer  graphics,  animation,  audio  and  video)  on  the  desktop.  These  systems  afford  instructional 
designers  the  capability  to  present  a  information,  real  problem  situations,  and  real  life  experiences 
(Bransford,  Sherwood,  Hasselbring,  Kinzer,  &  Williams,  1990).  On  the  theory  si  'e,  we  have  more  situated 
models  of  meaning  making.  Contemporary  approaches  to  instructional  design  are  more  concerned  with 
students'  abilities  to  use  the  knowledge  they  acquire  to  solve  real  life  problems.  Bransford  and  associates 
(Bransford,  Sherwood,  Hasselbring,  Kinzer,  &  Williams,  1990;  CTGV,  1990)  note  that  in  conventional 
teaching  and  learning,  the  students  often  fail  to  understand  the  value  of  new  information  for  problem 
solving,  because  they  do  not  experience  the  kind  of  problems  that  help  them  realize  how  information  can  be 
used  to  solve  meaningful  problems.  Therefore,  students  too  often  treat  new  knowledge  as  something  to  be 
memorized  which  results  in  the  acquisition  of  inert  knowledge.  Students  often  exhibit  oversimplifications 
and  compartmentalization  (Spiro,  Feltovich,  Jacobson,  &  Coulson,  199?N  Brown,  Collins  and  Duguid 
(1989)  also  argue  that  in  classroom  teaching,  teachers  attempt  to  teach  salient  features  to  the  students  and 
dismiss  the  contextual  and  peripheral  features  of  authentic  tasks  as  "noise."  This  decontextualization  does 
not  enable  learners  to  solve  real  life  problems.  "By  ignoring  ifie  situated  nature  of  cognition,  education 
defeats  its  own  goal  of  providing  usable,  robust  knowledge." 

A  number  of  models  for  designing  constructivism  learning  environments  have  been  proposed.  Most  of  them 
are  case-based  and  try  to  present  context-rich,  information-rich  and  situated  learning  environments  that  are 
relevant  to  learners  (Duffy  &  Jonassen,  1992.)  A  prime  example  of  case-based  learning  is  provided  by 
anchored  instruction  (CTGV,  1992;  CTGV,  1990;  CTGV,  1993),  which  provides  an  authentic  and 
generative  learning  environment  in  which  students  generate  and  combine  sub-goals  to  meet  the  challenges 
afforded  by  the  case.  Students  learn  to  use  mathematics  in  solving  the  problems,  rather  than  merely 
memorizing  formulas.  Very  rich  contextual  information  helps  the  project  achieve  this  goal. 

Brown,  Collins  and  Duguid  (1989)  argue  that  learning  is  indexed  by  the  experience  and  activities  in  which 
the  learning  happens.  They  argue  that  learning  should  be  situated  in  authentic  activities.  For  students  to  use 
tools  in  a  way  that  they  are  used  in  real  life,  students  must  "enter  the  community  and  its  culture,"  just  like 
a  craft  apprentice  learning  skills  from  a  master.  They  proposed  a  model  of  cognitive  apprenticeship,  which 
takes  into  account  the  situated  nature  of  leaning.  In  cognitive  apprenticeships,  the  emphasis  is  placed  on 
teaching  student  to  learn  how  experts  solve  problems  and  carry  out  tasks.  Cognitive  apprenticeship  is  a 
process  of  "learning-through-experience."  (Collins,  Brown,  &  Newman,  1989).  Making  coherent 
arguments  has  been  identified  as  one  important  outcome  of  instruction  in  constructivist  learning 
environments,  because  it  reflects  learners'  understanding  and  internal  organization  of  knowledge.  Most 
constructivist  approaches  to  learning,  such  as  cognitive  apprenticeships,  emphasize  a  different  set  of 
instructional  strategies  including  modeling,  coaching,  scaffolding,  articulation,  reflection,  and  exploration. 

Modeling  and  Coaching  of  Performance 

Of  particular  interest  to  this  study  are  two  support  strategies,  namely  cognitive  modeling  and  coaching. 
Bandura  (1977)  notes  that  by  providing  subjects  with  modeling,  performance  guidance,  corrective  feedback 
and  self-directed  mastery,  we  can  foster  learners'  skill  development  and  self-efficacy.  With  cognitive 
modeling,  the  teacher  can  expose  learners  to  the  expert's  covert  cognitive  processes  in  problem  solving. 
Usually  the  teacher  will  verbalize  the  internal  information  processing  and  reasoning  while  performing  the 
procedure^  involved  in  a  task.  By  experiencing  teacher's  cognitive  process,  students  are  better  able  to  adopt 
the  expert's  mode  of  thinking  (Gorrell  &  Capron,  1990). 

The  effectiveness  of  cognitive  modeling  has  been  demonstrated  in  many  research  studies.  Bruch  (1978) 
reported  two  experiments  supporting  the  use  of  cognitive  modeling.  Denney  (1975)  investigated  using 
cognitive  modeling  as  a  way  of  enhancing  children's  problem  solving  efficiency  in  a  question-asking  task. 
Three  kinds  of  strategies  were  studied:  cognitive  modeling,  watching  people  ask  questions,  and  self-rehearsal 
of  key  strategies.  Fildcr  children  benefited  from  all  treatments,  but  cognitive  modeling  was  the  strategy  that 
was  used  by  most  children  from  age  6  and  up.  In  mathematics,  students  with  difficulties  also  benefited  from 
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cognitive  modeling  and  guided  performance.  Making  the  covert  problem  solving  process  observable  for  the 
learners  by  giving  exemplary  modeling  and  explanation  of  internal  processes  proved  to  be  an  effective  way 
to  scaffold  students'  performance.  (Schunk,  1981,  Welkowitz  &  Calkins,  1984). 

Research  conducted  by  Englert  and  Raphael  (1988)  indicate  that  cognitive  modeling  and  coaching  are  also 
effective  strategies  for  supporting  expository  writing.  With  their  Cognitive  Strategy  Instruction  in  Writing 
program  (CSIW),  they  taught  special  education  students  writing  strategies  through  the  use  of  think-alouds 
that  model  underlying  writing  process.  In  this  program,  they  also  used  "think  sheets"  with  prompting 
questions  that  help  students  focus  on  audience,  purpose,  background  knowledge,  organizations  and  steps  of 
the  writing.  "Think  sheets"  also  serve  as  a  tool  to  help  student  think  through  the  writing  process 
reflectively.  Evaluation  results  indicate  positive  results  for  using  these  strategies  in  teaching  writing. 

It  is  worth  noticing  that  most  implementations  of  cognitive  modeling  has  been  provided  by  think-aloud 
protocols.  Students  experience  the  covert  and  internal  cognitive  processes  of  the  models  through  watching 
models'  performance  and  listening  to  their  speech  explaining  the  process  and  strategies.  With  today's 
computer  technology,  we  believe  that  it  is  possible  to  reveal  model's  cognitive  process  by  providing 
explanatory  descriptions  of  the  process  as  learners  attempt  to  solve  problems.  In  this  study,  we  explored  an 
experimental  implementation  of  coaching  and  modeling  in  a  computer  based,  case-based  learning 
environment.  In  this  study,  the  instructional  outcome  is  the  ability  to  make  an  argument  in  support  of 
purchasing  decisions.  In  order  to  make  sound  argumentation,  students  not  only  need  to  assimilate  the 
thinking  process  of  the  experts,  but  also  need  to  articulate  their  own  reasoning  and  thinking  in  a  way  that 
the  expert  would  do.  Cognitive  modeling  here  serves  as  both  model  of  reasoning  process  and  model  of 
desired  behavior  outcome. 

Individual  Differences  in  Using  Modeling  and  Coaching 

In  this  study,  we  attempted  to  investigate  the  interaction  between  the  instructional  treatments  and  the 
learning  characteristic,  locus  of  control.  Locus  of  control  describes  an  individual's  generalized  expectancy 
about  how  reinforcement  is  controlled,  either  by  internal  or  external  means.  Individuals  are  internally 
oriented  if  they  perceive  a  contingency  between  events  in  which  they  are  involved  and  their  own  actions. 
Such  indi-iduais  perceive  events  to  be  under  personal  control.  Individuals  who  feel  that  they  can  influence 
the  environment  will  actually  seek  ways  to  control  the  environment,  when  that  control  can  be  instrumental 
in  attaining  their  goals.  Externally  oriented  individuals  perceive  events  as  being  unrelated  to  their  behavior 
or  characteristics,  and  thereby  not  under  personal  control.  They  tend  to  attribute  the  outcome  of  events  to 
external  factors  (e.g.,  luck).  Personal  perceptions  of  causality  have  been  demonstrated  to  be  important 
mediators  in  many  situations.  As  a  psychological  attribute,  locus  of  control  has  been  linked  with  the 
independent  use  and  case-based  learning  environments. 

Lefcourt  (1982)  identified  some  of  the  cognitive  activities  in  that  demonstrated  differences  between  internals 
an^  externals  such  as  information  assimilation,  attention,  sensitivity  to  the  meanings  or  reinforcement 
opportunities  inherent  in  different  tasks  and  situations,  and  concentration.  He  concluded  Jiat  internals  were 
found  to  be  more  perceptive  to  and  ready  to  learn  about  their  surrounds.  They  are  more  inquisitive;  they  are 
more  curious  and  efficient  processors  of  information  than  are  externals.  There  is  a  trend  of  positive 
correlation  between  internality  and  academic  achievement.  Internals  should  be  more  adept  at  using  leamer- 
controlled,  case-based  learning  environments. 

Internal  oriented  individuals  are  more  exploring,  they  tend  to  be  more  comfortable  in  the  situation  in  which 
they  need  to  make  decisions.  Internals  should  be  better  arguers.  On  the  other  hand,  external  individuals  tend 
to  believe  that  their  effort  does  not  make  much  difference  in  the  outcome,  therefore,  they  often  rely  on 
external  lesson  structures.  They  do  not  want  to  make  decision  because  they  believe  that  makes  no  difference. 
Therefore,  it  would  be  more  beneficial  to  them  if  the  lesson  provides  more  supportive  instructional 
strategies  (Holloway,  1978;  Carrier,  Davidson  and  Williams,  1987). 


ERIC 


Purpose  and  Hypotheses 

The  major  purposes  of  the  present  study  are 

•  To  investigate  and  compare  the  effectiveness  of  two  instructional  strategies,  modeling  and  coaching 

on  helping  students  to  articulate  and  support  their  decisions  in  a  case-based  learning  environment. 
•To  compare  the  effectiveness  of  modeling  and  coaching  on  helping  student  address  essential  criteria 
in  order  to  make  sound  decisions. 

•  To  investigate  how  locus  of  control  influences  students  performance. 

This  study  tested  the  following  hypotheses  about  modeling  and  coaching: 

1.  Students  in  the  modeling  treatment  will  write  essays  addressing  more  of  the  criteria  than  student  in 

the  control  or  coaching  treatments. 

2.  Students  in  the  coaching  treatment  will  write  essays  addressing  more  of  the  criteria  than  student  in 
the  control  treatment. 

3.  Students  in  the  modeling  treatment  will  write  more  integrated  and  coherent  justifications  than  other 
students. 

4.  Students  in  the  coaching  treatment  will  write  more  integrated  and  coherent  justifications  than 
student  in  the  control  treatment. 

5.  External  control  students  will  benefit  more  from  the  modeling  and  prompting  than  internal  control 

students. 

6.  Students  who  are  more  comfortable  using  computers  perform  better. 


Method 
Subjects 

Fifty-five  college  students  from  two  classes  of  School  of  Restaurant  and  hotel  Management  at  a  large 
eastern  university  comprised  the  sample.  Sample  students  ranged  from  the  third  to  the  fifth  semester  in 
college.  Students  were  randomly  assigned  to  three  groups  by  their  instructor.  There  were  17  students  in  the 
modeling  group,  17  students  in  coaching  group  and  21  students  in  the  control  group.  None  of  the  students 
had  any  previous  experience  with  the  content  or  the  instructional  materials. 

Instructional  Materials 

The  instructional  material  was  a  computer-based  instructional  lesson  (with  three  different  versions)  which 
consisted  of  three  separate  restaurant  cases  that  required  learners  to  select  equipment  for  purchase  and  then 
justify  their  decision  based  upon  the  relevant  information  in  the  case.  Each  case  was  a  restaurant  under 
improvement.  Students  had  access  to  and  control  of  a  variety  of  information  in  each  case,  such  as  the 
location  of  the  restaurant,  the  requirements  and  expectation  of  the  restaurant  owner,  the  new  menu  structure 
and  menu  analysis,  specifications  of  different  pieces  of  equipment  to  choose  from.  They  also  could  access  a 
glossary  of  difficult  terms  and  use  a  built-in  calculator.  Their  task  was  to  write  a  report  describing  and 
justifying  the  kinds  of  equipment  to  buy  for  each  scenario.  These  decisions  and  justifications  were  recorded 
by  the  program. 

There  were  14  potential  criteria  that  each  student  should  think  about  before  making  the  justification.  These 
criteria  were  taught  explicitly  during  the  instruction. 

Treatment  1 :  Modeling  group.  In  addition  to  the  core  instructional  materials,  the  modeling  treatment 
provided  students  with  a  similar  case  scenario  with  think-aloud  statements  of  the  expert's  decision  on  which 
pieces  of  equipment  to  purchase  as  well  as  the  rationales  and  justifications  for  the  decision.  The  modeling 
scenario  was  presented  along  with  the  articulated  rationale1  and  justifications  in  a  field  next  to  the  window 
in  which  learners  were  required  to  w  rite  their  justification:*.  In  the  expert's  rationale,  the  reasoning  processes 
A-cre  articulated  and  the  criteria  to  be  considered  were  addressed.  This  modeling  section  was  available  to  the 
students  while  they  were  writing  their  own  decisions  and  rationales. 
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Treatment  2:  Coaching  group.  Students  in  coaching  group  also  had  the  core  instructional  materials.  Rather 
then  a  model  of  the  performance,  prompting  questions  were  presented  to  focus  their  writing  of  the  report. 
The  questions  prompted  the  students  to  think  about  the  critical  and  relevant  facts  and  the  criteria  when  they 
are  about  to  write  the  scenario.  Sample  questions  are:  "Have  you  gathered  all  the  relevant  facts  from  the 
case?  "  and  "Can  another  piece  of  equipment  perform  this  function  equally  well  or  better?" 

Students  in  both  treatments  could  return  to  the  case  scenarios  and  equipment  specification  database  from  the 
report  writing  section  in  order  to  obtain  information  to  support  their  decisions.  They  were  also  permitted  to 
revise  their  decisions  later.  The  modeling  and  coaching  were  again  available  lo  each  group  respectively 
when  they  returned  to  the  report  writing  section. 

Treatment  3:  Control  group.  Students  in  control  group  studied  the  core  materials  only  without  modeling 
and  coaching  in  report  writing  section. 

Procedure 

Three  groups  of  students  used  the  software  as  an  assignment  for  their  class.  A.11  versions  of  courseware 
were  stored  on  a  university  file  server  so  that  students  had  access  to  them.  To  help  students  to  get  the 
correct  version  of  courseware,  the  three  different  versions  were  given  different  names  and  icons.  Students 
finished  the  assignment  in  a  one-week  period  and  saved  experimental  data  on  floppy  disks.  The  courseware 
saved  version  information,  students  ID  and  their  reports.  Aiter  finishing  the  program,  data  were  collected  by 
gathering  data  files  recorded  on  students'  disks. 

Rotter's  Locus  of  Control  was  administered  after  the  assignment.  After  the  Rotter's  scale,  the  students  were 
required  to  use  a  5-point  scale  to  indicate  how  comfortable  they  felt  in  working  with  computers. 

Criterion  Measures 

Two  readers  scored  students' justifications  for  all  three  scenarios  on  two  scales.  For  each  rater,  the  first  scale 
is  used  to  assess  how  many  criteria  were  addressed  in  each  scenario.  This  scale  had  12  points  that  reflected 
the  12  criteria  to  be  considered.  Therefore,  each  student  had  3  criteria  scores.  The  average  criteria  score  was 
calculated  for  each  student.  The  second  scale  assessed  how  integrated  and  coherent  the  essays  were  in  regard 
to  the  use  of  given  information  and  the  underlying  reasoning  process.  This  scale  had  10  points  and  each 
student  had  3  coherence  scores  for  the  3  scenarios.  The  verage  coherence  score  was  also  calculated  for  each 
student.  In  judging  the  answers,  we  were  more  concerned  with  how  well  the  st  '°,nts  justify  their  decisions 
than  with  which  piece  of  equipment  they  decide  to  purchase. 

The  inter-rater  reliability  was  estimated.  The  correlation  between  average  criteria  scores  was  0.75  and  the 
correlation  between  average  coherence  scores  was  0.77.  In  statistically  analyzing  the  results,  the  average  of 
two  raters'  scores  were  used  as  indicators  for  coherence  scores  and  criteria  scores. 

Individual  Difference  Variable  Measure 

Rotter's  Locus  of  Control  Scale  (1966)  was  administered  to  each  subject  after  they  finished  the  lesson  as  a 
measure  of  the  locus  of  control.  Locus  of  control  describes  individual's  tendency  to  attribute  successes  and 
failures  to  internal  sources  such  as  effort  and  ability  or  external  sources  such  as  luck  pnd  fate. 
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Results 


We  use  criteria  scores  to  refer  to  the  average  scores  obtained  from  the  two  raters  on  how  many  of  the  12 
criteria  the  students  addressed  i  n  their  reports  and  we  use  coherence  scores  to  refer  to  the  average  scores 
obtained  from  the  two  raters  on  how  integrated  and  coherent  the  students'  reports  were. 

Table  1  presents  the  coherence  scores  achieved  by  the  three  groups,  and  Table  2  summarizes  the  analysis  of 
variance  (ANOVA).  The  ANOVA  revealed  statistically  significant  difference  on  coherence  scores,  F=46.49, 
P<.01. 


Table  1 

Coherence  scores  of  three  treatments 


Treatment 

N 

Mean 

SD 

Coaching 

17 

8.15 

0.51 

Modeling 

17 

8.21 

0.72 

Control 

21 

6.06 

1.01 

Table  2 

Analysis  of  variance  for  coherence  score  across  three  treatments 

Source 

SS 

DF 

Mean-Square 

F-  ratio 

P 

Treatment 

58.71 

2 

29.35 

46.49 

.00 

Error 

32.83 

52 

0.63 

A  Tukey  post  hoc  test  (p.<  .01)  indicated  that  students  performed  significantly  better  in  both  modeling  and 
coaching  groups  when  compared  with  the  control  group.  However,  there  was  no  statistically  significant 
difference  in  coherence  scores  between  modeling  and  coaching  groups  occurred. 


Table  3  presents  the  criteria  scores  across  treatments,  and  Table  4  summarizes  the  ANOVA.  No  significant 
difference  across  treatments.  F=0.148,  P<.86 

Measurement  of  locus  of  control  was  administered  after  the  instructional  treatments.  Because  some  of  the 
subjects  did  not  turn  in  the  questionnaire,  we  have  18,14,  and  13  subjects'  lo_us  of  control  data  for  control, 
coaching  and  modeling  group  respectively.  No  significant  differences  among  mean  scores  of  locus  of  control 
occurred  between  the  three  treatments. 


Table  3 

Statistics  of  criteria  scores  of  three  treatments 


Treatment 

N 

Mean 

L  SD 

Coaching 

17 

10.21 

1.70 

Modeling 

17 

9.86 

1.60 

Control 

21 

10.06 

2.14 

Table  4 

Analysis  of  variance  for  critci  ia  score  across  three  treatments 

Source 

SS 

DF 

Mean-Square 

F-ratio 

P 

Treatment 

1.01 

2 

0.51 

0.15 

.86 

Error 

117.90 

52 

3.42 

There  was  no  significant  correlation  between  locus  of  control  and  coherence  score.  r=-0.12,  p<.44. 
However,  the  correlation  between  locus  of  control  and  criteria  score  approached  significant  level.  r=-0.29, 
p<.()6.  The  correlation  between  locus  of  control  and  the  criteria  score  led  us  to  look  more  closely  into  these 
two  measures.  Modeling  group  had  the  least  correlation  between  locus  of  control  and  criteria  score.  A  linear 
regression  of  locus  of  control  and  criteria  score  for  Coaching  group  showed  a  multiple  R  of  0.29. 
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Table  5 


sis  of  variance  for  locus  of  control  score  across  three  treatments 


Source 

SS 

DF 

Mean-Square 

F-ratio 

P 

Treatment 

42.59 

2 

21.30 

1.27 

.29 

Error 

701.85 

42 

16.71 

Table  6 

Correlation  between  criteria  score  and  locus  of  control 


Group  Name 

N 

r 

P 

All  groups 

45 

-0.29 

.06 

Coaching 

14 

-0.36 

.21 

Modeling 

13 

-0.11 

.72 

Control 

18 

-0.29 

.23 

No  significant  correlation  between  the  self  rating  of  comfort  in  using  computers  and  criteria  score  and 
coherence  scores  occurred. 


Discussion 


Criteria  scores  indicated  how  many  criteria  students  addressed.  For  a  criterion  to  be  judged  as  addressed,  it 
only  needed  to  be  mentioned  in  the  report.  The  fact  that  all  of  the  12  criteria  were  taught  explicitly  made 
criteria  score  a  measure  of  recall  of  rules.  The  present  study  indicated  that  students  performed  equally  well  in 
all  three  groups.  Providing  additional  prompting  or  coaching  of  selections  criteria  did  not  help  students  at 
the  level  of  rule  recall. 

Coherence  scores  indicated  how  student  analyzed  the  problem  situation,  applied  the  knowledge  introduced  in 
the  courseware,  and  integrated  and  synthesized  their  solutions  to  the  problem.  Argumentation  and 
articulation  reflected  higher  level  of  thinking  process  and  real  life  problem  solving  skills  there  would  be 
needed  in  their  future  profession.  By  writing  purchasing  suggestions  and  rationale,  they  were  trying  to  speak 
the  lang  age  of  the  professionals  in  their  field.  The  result  of  present  study  suggested  that  both  coaching  and 
modeling  strategies  enhanced  students'  performance  in  writing  coherent  and  integrated  rationales  and 
argumentation.  However,  students  in  coaching  and  modeling  group  performed  equally  well,  indicating  that 
these  two  different  scaffolding  strategies  we're  equally  effective.  Our  current  implementations  of  both 
strategies  were  not  very  sophisticated  and  still  they  enhanced  performance  significantly  compared  to  that  of 
the  control  group.  However,  we  plan  how  students  will  perform  with  more  sophisticated  and  more 
powerful  implementations  of  coaching  and  modeling  in  an  upcoming  study. 

Locus  of  control  did  not  correlate  to  students'  coherence  scores.  However,  there  was  a  negative  correlation 
between  externality  of  locus  of  control  and  criteria  scores,  and  the  correlation  coefficient  was  approaching 
significance  (p<.06).  This  confirms  the  result  of  past  studies  that  internal  student  generally  perform  better 
than  external  students.  However,  initial  data  analysis  indicates  that  in  the  modeling  group  the  negative 
correlation  between  externality  and  coherence  was  least  significant.  The  scaffolding  afforded  by  modeling 
strategy  might  have  helped  external  students  keep  up  their  performance  with  that  of  internal  students. 
Because  of  the  relatively  small  sample  size,  further  study  is  needed  to  reach  a  more  certain  conclusion. 


Conclusion 


In  a  case  based  learning  environment,  students  can  make  more  coherent  and  integrand  argumentation  with 
either  coaching  strategies  or  modeling  strategies  a1,  scaffolds  of  performance.  Complex  responses  in  case- 
based  environments  can  be  supported  with  a  variety  of  strategies  aimed  at  getting  students  to  articulate  their 
thinking.  However,  neither  coaching  nor  modeling  helped  students  to  address  more  criuria  (have  bc'ter  rule 
recall).  Internal  students  outperformed  external  students.  However,  there  was  some  indication  that 
modeling  strategy  can  help  reduce  the  performance  difference  in  rule  recall  between  cxterral  and  internal 
students. 
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Background 

Widespread  change  is  dramatically  altering  the  traditional  face  of  the  workplace.  This  is  creating 
new  problems  and  conflicts  which  call  for  new  i  nterventions  and  strategies.  One  of  the  emerging  problems 
in  the  field  of  Instructional  Development  (ED)  is  the  widening  gap  between  theory  and  practice. 

Over  the  years,  the  majority  of  ED  studies  at  the  macro  level  have  focused  on  model  building 
(Andrews  &  Goodson,  1980;  Taylor  &  Doughty,  1988).  In  the  past,  theoreticians  identified  the  critical  ID 
component  tasks,  organized  them  in  a  logical,  linear  sequence  and  suggested  that  practitioners  follow  them 
systematically  and  thoroughly  (Gustafson  &  Tillman,  1991).  But,  this  strictly  step-by-step  procedure  does 
not  reflect  the  ID  process  as  practiced  in  the  Field,  since  it  is  what  works  that  counts  in  the  product-oriented 
corporate  world. 

Until  recently,  little  research  had  addressed  the  critical  issue  of  "what  works,"  let  alone  the  crucial 
decision-making  process  involved  in  corporate  ID  practice  or  the  quality  of  the  decisions  made.  ID 
professionals  may  have  to  explore  diverse  human  and  situational  factors  in  pursuing  this  line  of  research. 

In  examining  the  recent  developments  and  future  directions  of  ID,  the  literature  revealed  that  special 
attention  is  now  being  given  to  social,  political,  and  cultural  influences.  This,  in  turn,  is  leading  to  a  shift 
from  the  early  classical,  mechanistic  mode  to  a  more  flexible,  situational  and  social  approach  to  ED 
(McAlpine,  1992;  Schwen,  1988;  Stolovitch  &  Keeps,  1992;  Tessmer  and  Wedman,  1990  through  1993; 
Tripp,  etal.,  1990). 

In  addition,  a  review  of  the  literature  on  successful  corporate  ED  practice  provides  insightful 
evidence  that  the  ED  practitioner's  decisionmaking  process  is  profoundly  influenced  by  diverse  situational 
factors  (Gordon,  1986;  Greer,  1992;  Kaczmarek,  1992;  Rowland,  1992;  Steinburg,  1991 ;  Wedman  and 
Tessmer,  1993).  But  to  date,  the  process  of  identifying,  organizing,  weighing,  and  prioritizing  these 
factors,  as  well  as  the  process  of  making  decisions  and  visualizing  the  consequences  have  not  been  clarified 
by  research. 

Research  into  those  ID  processes  which  modify  theory  must  be  the  engine  which  drives 
professional  practice.  Exploring  both  the  ED  context  and  the  expert  practitioner's  decision-making  process 
in  the  corporate  setting  could  provide  insights  which  advance  ED  theory  and  give  practitioners  a  more 
realistic  set  of  options.  This  article  describes  the  two-year  naturalistic  inquiry  which  was  committed  to  this 
goal. 

From  the  researcher's  point  of  view,  this  study  proposes  a  conceptual  framework  which  will 
advance  ID  theory,  especially  at  the  macro  level  which  encompasses  planning,  development  and 
implementation.  From  the  ID  practitioner's  perspective,  it  explores  and  illuminates  the  influences  of 
critical  situational  factors  and  the  interaction  between  the  practitioner  and  the  surrounding  context.  This 
was  accomplished  by  asking  the  following  research  questions: 

•  What  are  the  critical  situational  factors  which  influence  the  corporate  ID 
practitioner's  decision-making  process? 

•  How  do  the  critical  situational  factors  influence  the  corporate  ID  practitioner's  decision-making 
process? 

•  What  patterns  exist  in  the  dynamic  interplay  among  the  critical  situational  factors  which  can 
elicit  a  descriptive  framework  for  what  may  be  done,  rather  than  a  prescription  for  what  must  be 
done? 

Situational  factors  in  this  study  are  defined  as  "a  variety  of  environmental  forces  which  influence 
the  ID  practitioner's  decision  making  in  a  specific  situation."  These  forces  may  act  either  as  restraints  or 
advantages  for  the  practitioner.  Five  situational  factors  were  identified  from  the  literature:  (I )  individual 
characteristics,  (2)  resource  availability,  (3)  client  needs,  (4)  organizational  influences,  and  (5)  market 
competition.  Market  competition  was  excluded  because  it  was  beyond  the  scope  of  this  study.  Three  new 
situational  factors  emerged  during  the  course  of  the  inquiry:  (a)  cultural  context,  (b)  nature  of  the  task,  and 
(c)  stakeholders.  Together,  these  seven  situational  factors  formed  the  core  of  this  study. 
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The  China  Productivity  Center  (CPC),  Taiwan,  was  chosen  as  the  setting  for  this  inquiry  for 
several  reasons.  First,  it  provides  a  concentrated  source  of  high  quality  corporate  consulting  and  training 
services  which  could  be  observed  in  situ.  Second,  CPC  carries  out  the  government's  policies  for  economic 
development  by  serving  the  nation's  small-  to  medium-sized  enterprises.  Third,  while  many  success  stories 
exist  in  Asia's  Pacific  Rim  countries,  Taiwan's  "economic  miracle"  is  one  of  the  very  best  examples. 
Finally,  for  corporations  seeking  to  internationalize,  Taiwan,  with  its  unique  location,  history  and 
economic  achievements,  serves  as  a  gateway  to  Asia;  and  for  Taiwan's  business  community,  CPC  serves  as 
a  gateway  to  the  world. 

In  this  article,  I  first  describe  the  research  method,  then  provide  a  summary  of  the  context  of  the 
study.  Next,  the  five  cases  of  the  four  ID  practitioners  are  presented  in  a  condensed  form.  This  is  followed 
by  a  discussion  of  research  findings.  Finally,  some  promising  areas  for  further  research  in  corporate 
instructional  development  are  identified. 

Methodology 

Heinich  (1984)  asserts  that  ID  research  efforts  should  focus  less  on  telling  practitioners  what  they 
should  do.  and  concentrate  instead  on  finding  out  what  the  conditions  are  that  shape  their  decisions.  By 
criticizing  the  laboratory  approach  as  not  compatible  with  instructional  management  realities,  Heinich  goes 
on  to  encourage  ID  researchers  to  engage  in  more  naturalistic  inquiry  (the  research  paradigm  as  described  by 
Lincoln  &  Guba,  1985).  The  present  study  followed  this  direction. 

Winn  (1986)  also  argues  that  it  is  imperative  that  the  case  study  method  be  considered  as  a  serious 
research  tool  for  ID  inquiry.  In  recommending  this  approach,  Winn  contends  that,  because  there  are  certain 
common  elements  in  every  ID  project,  "how  each  of  these  (elements)  is  handled  under  different 
circumstances  is  very  instructive,  and  usually  adaptable  to  other  projects"  (p.  350).  Therefore,  in  this 
inquiry,  the  qualitative  case  study,  with  its  emphasis  on  description  rather  than  prescription,  is  the  method 
chosen  to  cast  light  on  the  dynamic  interplay  of  critical  situational  factors  on  the  ID  practitioner's  decision- 
making process. 

Subjects 

Four  expert  ID  practitioners  were  selected  as  a  purposive  sample  of  subjects  for  this  study. 
Edward,  Mary  and  Emilie  are  Group  Managers  at  CPC.  Tony,  a  former  Group  Manager,  is  currently  a 
Senior  Manager  at  a  major  international  insurance  corporation.  Their  average  age  is  35  years;  their  average 
length  of  service  at  CPC  is  6.5  years. 

All  hold  bachelor's  degrees,  Edward  in  Mechanical  Engineering,  Mary  in  Law,  Tony  in 
Psychology,  and  Emilie  in  Graphic  Design.  Only  Emilie  has  a  formal  ID  education  background  (Master's 
in  Instructional  Technology).  Edward,  Tony  and  Emilie  hold  Human  Resources  Management  (HRM) 
Consultant  Certificates  issued  by  CPC,  while  Mary  has  had  extensive  work  experience  in  the  non-profit 
sector.  These  four  practitioners  started  their  professional  careers  either  as  trainers  or  training  coordinators. 
They  have  grown  professionally  and  are  well  on  their  way  to  completing  the  transition  from  ID 
practitioners  to  holistic  organizational  consultants. 

Data  Collection 

The  duration  of  this  study  was  January  1992  through  July  1994.  Preliminary  inquiries  were 
undertaken  both  in  the  United  States  and  at  the  China  Productivity  Center  (CPC)  in  Taiwan  between  July 
1992  and  December  1993.  Historical  and  observational  data  were  collected  during  this  period.  The  focus 
period  (January  1 994  through  April  1994)  consisted  of  an  in-depth  study  of  the  four  key  participants 
involve. 1  in  the  ID  process  at  CPC.  Post  data  analysis  and  report  writing  was  carried  out  between  May 
1994  and  July  1994. 

The  data  collection  techniques  used  were:  (1)  spontaneous  conversation,  (2)  interview  and 
discussion,  (3)  participant  observation,  (4)  field  notes  and  a  reflexive  journal,  and  (5)  document  and  record 
reviews.  The  interview,  discussion  and  participant  observation  proved  to  be  the  most  powerful  means  for 
revealing  the  way  the  practitioners  perceived  and  responded  to  reality. 


ERIC 


37 1   u  ^  0 


Informal  discussions  (e.g.,  those  held  during  the  process  of  developing  or  managing  the  ID 
projects)  often  provided  the  most  insightful  discoveries  of  the  "true"  socio-political  perspectives  of  the 
participants.  This  kind  of  discussion  together  with  semi-structured  interviews  served  as  the  primary  tools  of 
the  inquiry.  The  interviews  and  discussions  were  conducted  in  a  conversational  tone  to  preserve  the 
naturalness  of  the  interaction. 

Intensive  participant  observation  was  another  powerful  tool.  I  had  been  engaged  in  a  dialog  with 
these  subjects  and  with  CPC  as  an  organizational  entity  since  July  1992.  I  worked  side-by-side  with  the 
practitioners,  shared  their  frustrations  and  happiness,  their  struggles  and  celebrations  on  a  daily  basis. 
Participant  observation  thus  provided  me  with  an  in-depth,  here-and-now  experience.  It  allowed  me  to  live 
in  the  participants'  time  frame,  to  perceive  the  world  as  they  saw  it,  to  capture  their  emotional  reactions, 
and  to  confirm  what  was  being  discussed. 

Data  Analysis 

In  a  naturalistic  inquiry,  data  collection  is  carried  out  simultaneously  with  data  analysis.  This 
enables  the  inquirer  to  focus  and  to  „hape  his  or  her  study  as  it  proceeds.  But  in  this  article,  these  two 
processes  are  presented  separately  for  the  convenience  of  the  discussion.  A  combination  of  different 
methods  of  data  analysis  was  used.  These  included:  (a)  the  constant  comparative  approach  (Gloser  & 
Strauss,  1967),  (b)  the  modified  constant  comparative  approach  (Lincoln  &  Guba,  1985),  and  (c;  data 
analysis  strategies  recommended  by  Merriam  (1988). 

My  reflexive  journal  was  used  as  the  principal  tool  for  the  ongoing  data  analysis.  The  journal 
focused  on  (a)  those  issues  and  concerns  which  the  interviewees  addressed  most  centrally,  (b)  the  main 
themes  emerging  from  the  interviews,  discussions  and  observations,  and  (c)  any  speculations  or  guesses 
suggested  during  the  interviews,  discussions  and  observations. 

Post  data  analysis  activities  included  preparing  data  for  analysis  and  analyzing  for  category  and 
pattern.  In  preparing  dJ^  for  analysis,  all  of  the  information  collected  was  first  organized  into  a 
comprehensive  resource  package.  The  summaries  were  then  transferred  (together  with  any  pertinent 
quotations)  to  3X5  index  cards.  Each  card  carried  a  unit  of  information;  that  is,  any  relevant  statement  that 
stood  alone  without  further  emendation.  Each  card  was  then  assigned  three  codes  that  corresponded  to  the 
respondent's  first  initial,  the  date  of  the  mteraction,  the  ^age  number  and  sequence  on  the  summary  page 
(example:  T/2-8-94/3.2).  This  code  provided  a  means  by  which  all  data  used  could  be  located  within  the  raw 
data  sources. 

In  analyzing  for  categories  and  emerging  pattern,  I  began  by  sorting  the  cards  into  categories  which 
shared  common  properties,  and  then  gave  each  category  a  tentative  name.  I  cross-coded  and  duplicated  those 
cards  which  fitted  into  two  or  more  categories.  The  grouped  cards  were  re-sorted  and  assigned  a  formal 
category  name.  I  then  wrote  a  thematic  memo  about  each  group  of  cards.  These  thematic  memos  were 
compared,  analyzed  and  grouped  again  into  larger  patterns.  Finally,  I  established  theoretical  constructs  from 
the  distillation  of  the  patterns  which  emerged. 

Context 

In  the  past,  Instructional  Development  has  been  more  closely  aligned  with  education  than  with 
business.  The  ID  profession  is  growing  rapidlv  and  the  market  in  public  and  higher  education  has  become 
more  crowded.  An  increasing  number  of  practitioners  are  now  entering  the  corporate  world  as  trainers  and 
consultants.  The  majority  of  corporate  ID  practitioners  are  either  employed  by  private  companies,  serve  in 
government  agencies,  or  work  for  business  consulting  firms.  The  scenario  of  this  case  study  represents  a 
fourth  option.  The  China  Productivity  Center  in  Taiwan  is  a  government-business  partnership  for 
workforce  development. 

Taiwan 

Taiwan's  rapid  economic  development  has  brought  it  in  three  decades  from  an  agriculture  backwater 
to  the  13th  largest  trader  in  the  world.  More  recently,  it  has  drawn  international  attention  and  acclaim  for 
its  political  reforms.  Taiwan's  economic  and  political  liberalization  fostered  the  growth  of  pluralism,  which 

Or  ' • 
^  2  {) 


372 


in  turn  changed  social  structure  and  relationships.  High  rates  of  educational  attainment,  the  emergence  of  a 
vigorous  middle  class,  and  expanded  international  trading  networks  are  accelerating  Taiwan's  rate  of  social 
development. 

Under  great  pressure  to  keep  up  with  the  rapid  pace  of  change,  corporate  education  has  become  a 
business  strategy  to  stay  profitable  and  to  gain  a  competitive  advantage.  Many  corporations  in  Taiwan  send 
their  employees  to  an  outside  vendor.  While  numerous  private  consulting  firms  provide  training  services, 
some  government-business  partnership  are  also  heavily  involved  in  workforce  development.  The  China 
Productivity  Center  (CPC)  is  one  of  the  best-known  and  most  successful  examples. 

China   Productivity  Center 

CPC  was  established  by  the  government  in  1955  to  assist  Taiwan's  small-  and  medium-sized 
enterprises  in  improving  their  productivity  and  the  quality  of  their  services.  Run  as  a  non-profit 
government-private  parternship,  it  has  played  a  key  role  in  facilitating  Taiwan's  economic  growth. 

Since  1955,  CPC  has  carried  out  a  range  of  activities  which  fall  into  three  broad  categories:  (a) 
those  directly  connected  to  government,  such  as  implementing  productivity-related  policies;  (b)  those  in 
direct  response  to  client  needs,  such  as  providing  business  and  training  services;  and  (c)  support  services  for 
business  in  general,  such  as  forming  professional  associations,  producing  trade  and  professional 
publications,  establishing  productivity-related  university  programs,  participating  in  international 
organization,  and  facilitating  cocial  well-being. 

Under  the  leadership  of  Dr.  Casper  Y.  Shih,  CPC  has  undergone  three  critical  organizational 
changes  since  1984.  The  newly  decentralized  organizational  structure  has  created  a  climate  of  vitality  and 
autonomy.  Oniy  a  few  technical  professionals  at  CPC  have  ID  backgrounds-most  learn  by  self-study  and 
on-the-job  experience.  ID  practitioners  at  CPC  often  combine  roles:  organizational  consultant,  project 
manager,  activity  expert,  instructional  designer,  technical  expert,  and  administrative  support. 

Cases 

Following  is  a  summary  of  the  five  cases  of  the  four  experienced  ID  practitioners  studied  at  CPC 
in  Taiwan.  The  full  text  of  the  cases  can  be  found  in  the  original  research  manuscript  (Liang,  1994).  Some 
of  the  richness  of  the  interactions  which  took  place  during  the  inquiry  may  be  lost  in  this  present  form. 

The  first  case  describes  how  Edward  carried  out  a  government-sponsored  project  in  the  agricultural 
community.  Case  two  tells  the  story  of  Mary's  struggle  with  her  team's  transition  from  government- 
supported  to  profit-making  status.  The  third  introduces  Tony's  DWACT  corporate  ID  model.  Case  four, 
Tony's  second  case,  discusses  typical  problems  which  the  practitioner  might  encounter  in  corporate  ID 
work.  The  last  case,  Emilie's,  is  a  synthesis  of  many  of  the  features  from  the  previous  four  cases. 

Case  One  (Edward) 

Since  1971 ,  when  Taiwan  was  expelled  from  the  United  Nations,  the  dream  of  Taiwan's 
government  has  been  to  reenter  the  international  community.  To  this  end,  one  of  its  strategies  has  been  to 
participate  in  a  variety  of  international  organizations.  Becoming  a  member  of  GATT  (General  Agreement 
of  Tariffs  and  Trade)  is  just  one  of  many  examples.  But,  the  most  serious  internal  impact  of  participating 
in  GATT  is  that  local  agriculture  will  no  longer  have  the  government's  full  protection.  This  policy  is 
generating  a  great  deal  of  resistance  from  local  farmers. 

The  government  requested  CPC's  assistance  to  reduce  this  resistance  by  facilitating  a  policy  of 
discounting  the  production  and  marketing  costs  of  agricultural  products.  CPC's  mission  included 
introducing  the  local  farmers  to  contemporary  management  techniques,  developing  a  core  of  outreach 
administrative  profer-ionals,  and  disseminating  information  about  the  government's  agricultural  policies. 
Edward  was  assigned  as  Chief  Manager  for  this  project. 

The  way  '.n  which  Edward  implemented  this  project  differed  from  the  traditional  ID  process  both  in 
terms  of  content  and  approach.  He  provided  additional  on-site  consultations  and  arranged  supplemental 


media  promotion  events  to  support  the  content  of  the  activity.  He  chose  a  flexible,  cyclical  approach  to  the 
ED  process.  For  example,  when  he  grasped  the  essence  of  the  problem(s),  he  would  quickly  reach  a 
conclusion  and  then  "map  back"  the  whole  process  from  the  identified  conclusion. 

In  the  early  stage  of  implementing  the  training  activities,  Edward  focused  his  effort  on  the  fanners. 
But,  after  closely  observing  the  feedback  and  interactions,  he  decided  to  shift  his  focus  to  the  government 
agricultural  personnel  because  he  realized  that  the  latter  was  his  real  client.  In  other  words,  he  wanted  to 
channel  his  energy  where  it  would  have  the  most  impact. 

According  to  Edward,  the  instructors  in  this  project  were  mostly  CPC  in-house  professionals. 
This  intensive  involvement  was  motivated  by  several  considerations:  reducing  potential  resistance  from  the 
farmers  to  the  follow-up  consulting  services,  developing  more  CPC  experts  in  this  business  line,  and 
creating  a  market  niche  and  repeat  business  for  CPC. 

Edward's  wide  business  network  enabled  him  to  draw  in  a  number  of  regional  professionals  to  carry 
out  this  project.  He  and  his  team  established  several  sites  to  disseminate  the  government's  policies  and  to 
demonstrate  their  tangible  accomplishments.  The  high  visibility  of  the  project  results  facilitated  his  client's 
buy-in. 

Edward  identified  two  major  difficulties  which  he  had  to  overcome  during  the  project.  The  first 
was  related  to  the  unique  nature  of  agriculture,  e.g.,  farmers  tend  to  have  a  conservative  midset;  their  average 
level  of  formal  education  is  low;  they  need  to  work  during  daylight  hours.  This  influenced  the  timing  of 
his  team's  activities  and  the  way  they  communicated.  The  second  was  related  to  the  mindset  of  some  CPC 
staff  v  ho  tended  to  think  of  agriculture  as  a  low-end  business.  Edward  now  plans  to  capitalize  on  this 
experience  to  develop  an  Agricultural  Enterprise  Management  Consultant  program  to  foster  outreach 
professionals-in  anticipation  of  promising  market. 

Case  Two  (Mary) 

Group  A  was  formed  at  CPC  in  1988  to  promote  Quality  awareness.  The  group  held  annual 
Quality  Month  activities  which  usually  included  an  opening  convention,  seminars  on  a  topic  chosen 
annually,  various  media  promotions,  as  well  as  visits  to  exemplary  companies.  Because  Quality  month  had 
been  sponsored  by  the  government,  many  CPC  people  believed  that  Group  A  was  unable  to  support  itself 
financially. 

In  mid  1993,  Group  A  encountered  a  major  challenge.  The  government  decided  that  its  Quality 
awareness  policy  had  been  successfully  diffused  throughout  Taiwan's  business  community,  and  discontinued 
the  group's  annual  supporting  budget.  Suddenly,  the  original  mission  of  Group  A  was  lost.  Mary,  the 
Group  Manager,  faced  a  tough  choice:  Whether  or  not  to  keep  her  group  going. 

Mary  quickly  held  several  meetings  with  her  team  members  tc  announce  this  discouraging  news, 
to  exchange  opinions  and  to  consolidate  the  consensus.  She  simultaneously  met  with  CPC  top 
management  to  proclaim  her  team's  intention  and  commitment,  and  to  gain  inter-organizational  support. 

After  reaching  a  consensus  and  gaining  team  and  management  support,  Mary  did  two  things. 
First,  she  planned  the  next  year's  activities  from  their  zero  income  base.  Second,  she  took  action  to  handle 
her  team's  transition  both  psychologically  and  behaviorally.  To  shift  from  its  original  "non-profit"  identity 
to  a  "making-a-reasonable-profit"  one,  Mary  led  her  team  to  identify  what  should  be  kept  and  what  should 
be  discarded.  This  shift  involved  a  new  mindset  and  subsequent  behavior  patterns. 

During  this  tough  time,  Mary  worked  with  her  team  members  to  experiment  with  the  new 
approach.  Experience  was  accumulated.  New  ideas  and  strategies  bloomed  and  were  carried  out.  As  a 
result,  the  Quality  Club  and  the  Planning  Professional  seminar  became  two  examples  of  the  new  ideas 
which  bore  fruit. 

The  Quality  Club  is  an  association  of  diverse  corporations  interested  in  quality  issues.  The 
strategy  behind  forming  this  Club  was  to  ensure  a  basic  business,  to  establish  a  communication  channel 
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with  clients,  and  to  create  an  environment  for  enterprises  to  establish  information  networks  and  to  form 
alliances. 

The  Planning  Professional  seminar,  on  the  other  hand,  was  a  strategy  to  build  the  team  members' 
professional  competencies.  Mary  developed  this  education  activity  by  mimicking  what  she  usually  does. 
Believing  that  the  process  for  many  things  is  similar,  Mary  stressed  that  the  continuous  experience  of 
communication,  negotiation,  implementation,  ar      vision  is  a  necessity,  because  corporate  situations  are 
always  in  a  state  of  flux. 

To  achieve  successful  project  outcomes,  Mary  used  three  key  strategies:  (a)  knowing  the  critical 
success  factors  for  different  kinds  of  activities,  (b)  distinguishing  "our  success"  from  "our  client's  success," 
and  (c)  using  the  client's  standards  as  her  criteria. 

Mary  encountered  two  particularly  difficult  problems.  The  first  was  the  need  of  providing  a  clear 
vision  for  her  team,  because  she  could  not  guarantee  the  team  members  anything.  People  had  to  overcome 
the  anxieties  about  survival  before  they  could  concentrate  on  the  mission.  Performance  evaluation  was 
Mary's  second  problem.  Quality  awareness  activities  were  large  in  scale  and  abstract  in  content,  which 
made  their  achievements  difficult  to  measure  quantitatively. 

Case  Three  (Tony  I) 

Fleet  is  a  major  corporation  in  Taiwan's  automobile  industry.  Bill,  the  President,  wanted  to 
globalize  his  business.  One  of  his  strategies  was  to  enhance  Fleet's  human  resources  with  CPC's 
assistance.  Tony  conducted  a  series  of  human  resources  development  projects  for  Fleet,  which  became 
known  as  the  Fleet  Project. 

The  Fleet  Project  continued  for  three  years  under  Tony's  leadership.  In  the  fourth  year,  following  a 
major  internal  re-organization  at  CPC,  it  was  transferred  to  another  newly-formed  group.  Within  a  few 
months  of  this  "organizational  earthquake,"  the  new  group  encountered  serious  turnover  problems  and  the 
project  was  terminated. 

Before  Tony  took  on  the  Fleet  project,  he  considered  several  issues  from  CPC's  stand  point:  Was 
this  project  for  "survival"  or  for  "development"?  Would  it  create  a  market  niche?  Was  he  mentally  ready 
for  this  project?  Was  he  professionally  equipped  to  handle  this  project?  Were  the  resources  adequate?  and, 
How  tough  was  the  client? 

Although  Tony  stressed  that  the  Fleet  Project  owed  its  success  to  the  strong  support  from  top 
management,  it  was  his  effective  approach  which  formed  the  backbone  of  the  accomplishment.  Tony  used 
his  experience  with  Fleet  to  develop  the  DWACT  corporate  education  model  (Diagnosis,  Workshop, 
Action,  Consultation,  and  Training).  DWACT  is  an  in-house,  tailor-made  corporate  education  service 
model  (see  Table  1).  Although  the  five  steps  in  the  DWACT  model  are  presented  in  a  procedural  format, 
the  practical  operation  of  this  model  is  an  overlapping,  concurrent  approach  (see  Figure  1). 

Clearly,  what  differentiates  the  DWACT  model  from  the  traditional  ID  approach  is  its  emphasis  on 
action,  the  use  of  non-educational  interventions,  the  consultation  activity,  and  the  concurrent  pattern  of 
activities.  According  to  Tony,  the  three  primary  functions  of  this  model  are:  to  amplify  the  value  and 
effectiveness  of  educational  activities;  to  highlight  the  market  differentiation;  and  to  bring  an  impact  to  the 
client  organization. 

In  the  diagnosis  process,  Tony  used  the  standard  business  technique  of  SWOT  (Strength, 
Weakness,  Opportunities,  and  Threats)  to  identify  the  client's  problems,  and  at  the  same  time  he  passed  on 
his  diagnostic  skills  to  the  client.  Before  carrying  out  the  workshop  presentation,  Tony  discussed  his 
agenda  with  the  client  representative.  He  found  and  then  got  rid  of  those  issues  which  conflicted  with  the 
Fleet  organizational  culture.  Tony  then  designed  the  workshop  handouts  in  several  modules  which  could  be 
combined  according  to  the  different  needs  of  the  participants.  The  client  then  executed  Tony's  recommended 
action  plan.  Tony  and  his  team  provided  consulting  help  when  requested.  The  training  activities  were 
conducted  concurrently  with  the  action  deployment. 
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Tony's  lack  of  familiarity  with  his  client's  core  technology  [specific  industry,  issues  and 
terminology]  was  the  major  problem  in  the  Fleet  project.  But,  Tony  indicated  that  this  was  not  difficult  to 
solve  because  CPC  has  accumulated  a  wealth  of  consulting  experience  over  time.  Thus,  the  practitioner's 
ability  to  maximize  CPC's  resources  becomes  central  to  the  problem. 

Table  1:  The  DWACT  Corporate  ID  Model  (Source:  Internal  CPC  document,  Pang,  1992) 


DWACT  Corporate  ID  Model 


Diagnosis 

•  Collect,  categorize,  and  analyze  the  client's  needs  and  problems  via  interviews  and/or  questionnaires. 

•  Design  and  structure  the  activity  deployment  by  tracing  the  desired  outputs  back  to  inputs. 

•  Present  the  findings  to  the  client  and  revise  thern  based  on  the  client's  feedback. 

Workshop 

•  Arrange  the  necessary  contextual  considerations  into  the  workshop. 

•  Stimulate  the  participants'  discussion  via  instruction,  information  sharing,  simulation,  group  exercise, 
or  by  changing  "the  rules  of  the  game." 

•  Consolidate  the  participants'  consensus  and  commitment,  as  well  as  construct 
an  action  plan. 

Action 

•  Deploy  the  action  plan  in  concert  with  a  cognitive  change. 

•  Coordinate  and  evaluate  the  action  deployed,  then  present  the  performance  outcomes  to  the  client. 

Consultation 

•  Assist  in  improving  the  action. 

•  Facilitate  performance  by  giving  feedback  and  advice. 

Training 

•  Re-equip  and  upgrade  techniques  necessary  for  the  ongoing  actions. 

•  Enhance  the  background  knowledge  and  skills  of  participants  for  the  follow-up  activities. 


Diagnosis 

Workshop 

Action 
Consultation 
Training 


Figure  1:  The  Process  Model  of  DWACT 
Case  Four  (Tony  II) 

Informatics  is  a  renowned  government-sponsored  research  and  development  institute.  Its  main  duty 
is  to  conduct  R&D  projects  which  promote  the  development  of  Taiwan's  strategic  industries.  In  addition. 
Informatics  helps  local  hi-tech  industries  with  their  research  activities.  Because  of  its  unique  mission  and 
characteristics,  most  of  its  operations  must  adhere  strictly  to  government  policies  and  regulations.  Many 
professionals  at  Informatics  had  complained  about  the  cumbersome  purchasing  process  and  the  complexity 
of  its  project  contracts.  Informatics  contracted  with  CPC  to  solve  these  problems.  Tony  was  eager  to  take 
on  this  project  because,  if  it  succeeded,  it  would  bring  CPC  great  credibility.  But,  he  also  worried  that 
Informatics'  high  expectations,  its  lack  of  experience  working  with  ID  practitioners  and  its  internal  politics 
would  hinder  his  accomplishment. 

After  an  in-house  diagnosis,  Tony  concluded  that  Informatics'  problems  stemmed  both  from 
operational  and  policy-related  causes.  Although  there  was  adequate  room  for  improvement  on  the 
operational  side,  it  would  be  difficult  to  get  a  radical  improvement  on  the  policy  side  due  to  the  inflexibility 


of  the  regulations.  But,  the  client  could  not  grasp  the  complexity  which  the  project  might  involve. 
Because  of  the  political  considerations,  instead  of  taking  a  holistic  approach  to  these  problems,  Tony 
decided  to  follow  the  client's  ill-defined  intention,  and  focus  only  on  improving  the  operational  process. 

In  addition  to  the  difficul'ies  identified  above,  many  other  problems  arose  during  this  project.  The 
client's  problematic  internal  cross-functional  communication  and  a  lack  of  top  management's  support  were 
two  major  roadblocks.  The  problems  associated  with  the  practitioner  side  were:  (1)  the  hidden  agenda  (Tony 
originally  planned  to  transfer  the  leadership  of  this  project  to  Joe,  another  CPC  professional);  (2) 
unexpected  organizational  change  (Joe  was  assigned  to  lead  a  newly-formed  group);  and  (3)  the  lack  of 
knowledge  about  the  client's  industry. 

Tony  re-designed  both  the  purchasing  and  contracting  processes  and  trained  the  institute  staff 
involved  in  the  new  working  procedures.  As  a  result,  the  cycle  time  of  the  operational  processes  was 
reduced  by  40%,  but  the  policy-related  procedures  were  not  improved.  By  examining  the  "whole  task,"  the 
impact  was  not  so  significant.  The  client  criticized  CPC  team's  performance.  Tony  felt  powerless  and 
frustrated. 

Case  Five  (Emilie) 

Knowseeds  is  a  major  corporation  in  Taiwan's  telecommunications  industry.  It  contracted  with 
CPC  to  introduce  the  Kaizen  business  philosophy  and  techniques.  Top  management  at  Knowseeds  was 
highly  committed  to  this  effort.  But,  this  commitment  resulted  in  extra  pressure  on  the  Knowseeds 
employees,  a  pressure  which  demanded  visible  results.  Michael  (Manager  of  Department  C  at  Knowseeds 
and  representative  to  CPC)  felt  the  most  stress.  Not  only  did  the  pressure  come  from  Knowseeds  as  a 
whole,  but  from  his  own  division,  because  Department  C  had  long  enjoyed  a  reputation  for  high 
performance. 

However,  by  its  nature.  Kaizen  is  not  a  business  strategy  which  lends  itself  to  fast  success  or  quick 
fixes.  And,  Department  Cs  consistent  history  of  high  performance  left  it  little  room  for  improvement. 
Thus,  its  opportunity  for  impressive  results  was  much  smaller  than  that  of  other  departments  with  lower 
performance.  Moreover,  at  the  beginning  of  this  project,  Knowseeds  incurred  some  inevitable  start-up  costs 
and  the  employees  were  required  to  institute  economics  in  their  customary  operations.  Subsequently,  a 
variety  of  problems  arose. 

Dealing  with  the  Knowseeds  employees'  reactions  was  the  first  problem.  These  workers  had  to 
change  their  operating  procedures  when  quality  techniques  were  introduced.  But,  the  profits  gained  from  this 
change  were  not  credited  to  the  efforts  of  these  front-line  "Kaizen-implementers."  They  felt  exploited.  The 
second  problem  was  related  to  investment-return.  Businessmen  usually  look  for  a  fast  return  on  their 
investment.  This  is  reflected  in  their  attitude  when  contracting  for  training  or  consulting  services. 

The  third  problem  was  the  practitioner's  unfamiliarity  with  the  social  dynamics  of  the  client 
system,  particularly  the  interpersonal  networks,  hidden  agendas,  readiness  for  change,  resource  allocation  and 
response  to  evolving  situations.  Additionally,  Emilie  wrestled  with  the  values  and  nature  of  her  profession 
by  asking:  What  is  your  professional  aim?  Who  exactly  do  you  want  to  help  in  your  profession?  What 
and  how  do  you  want  to  contribute  to  this  society?" 

A  couple  of  months  after  initiating  this  project,  Michael  (the  Knowseeds'  manager)  became 
impatient  at  the  delay  in  outcomes.  He  did  not  want  this  project  to  hurt  his  reputation.  He  began  to 
dispute  certain  issues  with  Emilie  and  her  team  members.  Emilie  found  that  Michael  often  chose 
approaches  which  would  produce  immediate  and  visible  results,  but  which  might  not  lead  his  department  or 
Knowseeds  to  a  better  future.  As  the  tension  rose,  Michael's  management  style  changed,  gradually  shifting 
from  a  coach  to  a  monitor  of  his  subordinates. 

In  this  project,  Emilie  decided  to  lead  the  client  organization  to  reform  its  related  policies  and 
regulations,  e.g.  the  incentive  system.  She  wanted  to  ensure  that  productivity  was  rewarded  and  to  prevent 
devious  employees  from  taking  advantage  of  the  fluid  situation. 
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Discussion  of  Findings 


These  five  cases  represent  the  initial  effort  to  distill  the  raw  data  collected  during  the  inquiries.  The 
follow-up  distillation  of  the  data,  which  integrates  the  research  questions  and  literature  review,  draws  out  the 
critical  situational  factors  which  were  found  to  influence  the  practitioners'  decision  making.  The  findings 
are  then  formed  the  subsequent  theoretical  constructs.  In  this  section,  the  findings  of  this  inquiry  are 
discussed  under  four  themes:  (a)  situational  factors,  (b)  interlacing  network,  (c)  decision-making  process,  and 
(d)  corporate  ID  practice. 

Situational  Factors 

Five  situational  factors  in  the  ID  process  were  initially  identified  from  the  current  literature.  These 
were:  (1)  individual  characteristics,  (2)  resource  availability,  (3)  client  needs,  (4)  organizational  influences, 
and  (5)  market  competition.  Market  competition  was  excluded  because  of  the  limited  scope  of  the  study. 
During  the  course  of  this  inquiry,  three  additional  situational  factors  emerged:  (a)  cultural  context,  (b)  nature 
of  the  project,  and  (c)  stakeholders.  These  three  new  factors  were  combined  with  the  original  four,  forming 
a  core  of  seven  situational  factors  which  are  addressed  below  (see  Table  2). 


Table  2:  Key  Situational  Factors  in  Corporate  ID  Practice  (from  research  on  "The  impact  of 

situational  factors  on  the  corporate  instructional  development  practitioner's  decision  making,"  Liang,  C,  1994) 


Key  Situational  Factors  in  a  Corporate  ID  Project  | 
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•  Client's  Needs 

The  goals,  expectations,  intentions,  standards,  agendas,  policies,  and  the 
specific  demands  of  the  industry  of  the  client. 

•  Cultural  Context 

The  specific  environment  in  which  the  ID  project  is  carried  out  under  a  set 
of  shared  cultural  traditions,  values,  beliefs,  arid  the  influences  of 
contemporary  social  issues  and  concerns. 

•  Organizational  Influences 

The  effects  of  the  mission,  values,  culture,  structure,  and  politics  of  the 
practitioners'  system  on  the  operations  and  outcomes  of  the  project. 

•  Individual  Characteristics 

The  academic  background,  work  experience,  personal  values,  status  in  the 
organization,  professional  competencies,  and  interpersonal  networks  of  the 
practitioner. 

•  Resource  Availability 

The  personnel,  budget,  time,  materials,  facilities,  information  and  other 
resources  necessary  to  implement  the  project. 

|  •  The  Nature  of  the  Project 

The  tasks,  scope  and  priorities  of  the  project  handled  by  the  practitioner. 

|  •  Stakeholders 

The  needs,  concerns  and  benefits  of  those  participants  who  have  an 
interest  in,  or  affected  by,  the  activities  and  outcomes  of  the  project. 
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The  client's  needs  were  found  to  be  the  most  critical  of  the  seven  situational  factors  in  the 
experienced  practitioners'  decision  making.  The  factors  of  cultural  context,  organizational  influences, 
individual  characteristics,  and  resource  availability  were  critical  in  all  five  cases.  The  nature  of  the  project 
and  stakeholders  were  less  critical  but  still  significant. 

This  study  demonstrated  the  primary  importance  of  the  client's  needs  in  the  practitioner's  decision 
making.  Each  of  the  four  practitioners  put  their  clients'  needs  as  their  first  priority  and  took  their  clients' 
tangible  requests  or  vague  hints  seriously  in  order  to  ensure  the  projects'  success,  stay  in  business,  and 
make  a  profit. 

The  cultural  context  was  the  first  of  the  three  new  situational  factors  found  in  this  study. 
Localizing  materials  was  an  example.  To  localize  means  to  get  rid  of  or  to  modify  some  notions  which 
conflict  with. local  culture  and  to  add  regional  examples  in  order  to  make  sense  to  the  local 

audience.  Many  of  CPC's  educational  materials  are  imported  from  Japan  or  the  United  States.  The 
practitioners  translated  and  then  customized  these  materials  to  match  their  clients'  needs. 

The  client's  needs  were  found  to  be  the  most  critical  of  the  seven  situational  factors  in  the 
experienced  practitioners'  decision  making.  The  factors  of  cultural  context,  organizational  influences, 
individual  characteristics,  and  resource  availability  were  critical  in  all  five  cases.  The  nature  of  the  project 
and  stakeholders  were  less  critical  but  still  significant. 

This  study  demonstrated  the  primary  importance  of  the  client's  needs  in  the  practitioner's  decision 
making.  Each  of  the  four  practitioners  put  their  clients'  needs  as  their  first  priority  and  took  their  clients' 
tangible  requests  or  vague  hints  seriously  in  order  to  ensure  the  projects'  success,  stay  in  business,  and 
make  a  profit. 

The  cultural  context  was  the  first  of  the  three  new  situational  factors  found  in  this  study. 
Localizing  materials  was  an  example.  To  localize  means  to  get  rid  of  or  to  modify  some  notions  which 
conflict  with  local  culture  and  to  add  regional  examples  in  order  to  make  sense  to  the  local  audience.  Many 
of  CPC's  educational  materials  are  imported  from  Japan  or  the  United  States.  The  practitioners  translated 
and  then  customized  these  materials  to  match  their  clients'  needs. 

Organizational  influences  repeatedly  proved  to  be  significant  throughout  all  five  cases  and  had  a 
profound  impact  on  each  practitioner's  daily  operations.  This  factor  was  also  a  major  source  of  the 
practitioners'  professional  ethics.  Organizational  influences  flowed  through  to  the  client  systems  by  way  of 
CPC's  triadic  business  strategy  of  kaizen,  innovation  and  anticipation. 

The  factor  of  individual  characteristics  was  also  significant  in  all  five  cases.  In  addition  to  the 
practitioners'  professional  competencies,  other  aspects  of  individual  characteristics  were  especially 
noticeable:  in  Edward's  case  it  was  his  interpersonal  relationships;  in  Mar/;  case  it  was  her  previous  non- 
profit work  experience;  in  Tony's  cases  it  was  his  business  capacities;  and  in  Emilie's  case  it  was  her 
personal  values. 

Resource  availability  was  a  significant  factor  in  Tony's  Fleet  project  and  in  Mary's  transition  case, 
but  for  different  reasons.  While  Tony  was  amply  supported  by  his  client,  Mary  had  lost  her  major  source  of 
financial  support.  Resource  availability  was  not  usually  a  major  concern  to  these  four  practitioners, 
because  funding  for  many  of  their  projects  came  either  directly  or  indirectly  from  the  government.  In  fact, 
these  experienced  practitioners  had  also  worked  hard  to  become  "resource-smart"  people.  Thus,  the  novice 
practitioner  who  wishes  to  become  "resource-rich"  should  become  "resource-smart"  by  first  becoming 
"resource-conscious". 

The  nature  of  the  project  emerged  as  the  second  new  situational  factor  which  has  significant  impact 
on  the  practitioner's  decision  making.  For  example,  the  complicated  nature  of  the  contracts  in  Tony's 
Informatics  case  and  the  long-range  timeframe  of  the  Kaizen  strategy  in  Emilie's  Knowseeds  case  were 
examples  which  demonstrated  this  influence. 


Stakeholders  emerged  as  the  third  new  situational  factor.  For  example,  in  Edward's  case,  it  was  the 
characteristics  of  the  farmers  (e.g.,  their  education  level)  which  influenced  his  decisions  on  the  design  of  the 
training  materials  and  the  methods  of  delivering  and  communicating  concepts.  The  opinions  and  concerns 
of  the  other  stakeholders,  the  agricultural  administrators  and  scholars,  also  had  determining  effects  on  his 
pioject  decisions.  Emilie's  concerns  about  the  reactions  and  welfare  of  the  Knowseeds  employees  was 
another  example  of  the  emerging  focus  by  practitioners  on  stakeholders'  needs  and  benefits. 

Interlacing  Network 

A  pattern  was  evident  in  the  dynamic  interplay  among  the  seven  situational  factors  which  emerged 
in  all  five  cases.  Throughout  each  project,  these  factors  influenced  each  other  and  developed  their  own 
sophisticated,  interlacing  network.  Each  factor  had  its  links  to  the  others.  Some  of  these  links  were  solid, 
while  others  were  subtle.  It  was  the  immediacy  of  the  relationship  that  made  the  links  solid.  The  subtle 
links  became  solid  in  a  specific  instance  when  a  close  relationship  occurred.  Figure  3  proposes  a 
construction  of  this  interlacing  network  of  situational  factors. 

The  factor  of  individual  characteristics  is  used  here  as  an  example  to  explain  the  nature  of  the 
linkages  among  the  situational  factors.  The  link  between  individual  characteristics  and  cultural  context  is 
solid  because  Taiwan  society  is  where  these  practitioners  grew  up,  live  and  work.  Resource  availability,  is 
directly  influenced  by  the  quality  of  the  practitioners'  interpersonal  relationships.  Organizational  influences 
(CPC's  unique  mission,  values,  culture  and  leadership)  stimulate  their  thoughts  and  guide  their  professional 
conduct.  Their  personal  traits  and  professional  competencies  determine  if  these  practitioners'  capacities 
match  their  clients'  needs  and  whether  or  not  they  are  likely  to  fulfill  their  clients'  goals. 




Cultural 
Context 


Figure  3:  Interlacing  Network  of  Situational  Factors  (from  research  on  "The  impaci  of 

situation;  !  factors  on  the  corporate  instructional  development  practitioner's  decision  making,"  Liang,  C.  1994) 


Usually,  the  factor  of  individual  characteristics  does  not  have  a  close  association  with  the  nature  of 
the  project  or  the  stakeholders'  benefits.  But,  in  Mary's  team  transition  case,  individual  characteristics  and 
the  nature  of  the  project  did  become  closely  related.  Because  Mary's  job  had  been  non-profit  in  nature,  this 
"habitual  domain"  psychologically  hindered  her  professional  practice  during  her  team's  transition  to  self- 
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supporting  status.  In  Emiiie's  Knowseeds  case,  the  link  betw  n  individual  characteristics  and  stakeholders 
became  solid  because  of  her  personal  values.  In  asking  who  exactly  a  corporate  ID  practitioner  wants  to 
help,  she  was  expressing  concern  about  the  Knowseeds  employees'  reactions  and  benefits. 

As  the  data  emerged,  the  interlacing  network  of  situational  factors  proved  to  be  an  important 
construction  which  could  track  the  practitioners'  decision-making  processes.  This  network  showed  how 
they  analyzed  the  causes,  current  status  and  developmental  directions  of  the  problems  they  encountered.  It 
provided  a  mental  framework  which  could  be  used  to  visualize  possible  actions  and  the  resulting 
consequences,  and  to  create  a  prioritized  list  of  the  action  events. 

Decision-Making  Process 

Each  of  these  experienced  practitioner's  decision-making  processes  were  greatly  influenced  by  a 
variety  of  external  physical,  intellectual  and  social  forces  existing  in  the  surrounding  context.  It  was  the 
interplay  of  these  forces  which  formed  each  specific  project  situation.  The  practitioners  then  made  their 
decisions  based  on  their  interpretation  of  each  situation  and  its  context. 

When  the  practitioners  encountered  a  decision  point,  they  explored  the  problem  which  had  brought 
them  to  this  decision  point  and  delineated  the  readily  identifiable  situational  factor(s)  relevant  to  the  problem 
at  that  time.  They  then  mentally  analyzed  the  sophisticated  relationships  among  the  specified  factor(s)  and 
the  other  situational  factors. 

The  practitioners  constancy  updated  their  knowledge  about  the  problem  by  seeking  out  new 
information  from  the  context  and/or  searching  for  patterns  which  matched  their  previous  experiences.  They 
took  account  of  the  subsequent  change  in  the  relationships  among  the  situational  factors. 

This  continuous  interaction  with  the  situation  enabled  the  practitioners  to  move  gradually  from 
involvement  in  each  single  situational  factor  to  a  consideration  of  the  entire  relationship;  from  an 
exploration  of  the  problem  to  a  commitment  to  the  situation  (refer  to  Figure  3).  In  short,  they  held  a 
reflexive  conversation  with  the  situation. 

Bearing  the  project's  goal  in  mind,  the  practitioners  then  used  their  renewed  knowledge  to  construct 
possible  courses  of  action  and  to  reason  out  possible  consequences  for  each  of  these  actions.  Subsequently, 
the  practitioners  mentally  developed  a  prioritized  list  of  action  events  based  on  their  intuition  and/or 
judgment  by  visually  rehearsing  and  comparing  possible  combinations  of  actions  and  outcomes.  They  then 
implemented  the  action(s)  they  had  selected  by  following  that  priority. 

Throughout  their  projects,  the  practitioners  continued  the  processes  of  seeking  out  new 
information  and  searching  for  patterns  in  prior  experiences  to  assess  the  emerging  situations.  Or,  they 
moved  in  a  similar  way  to  address  the  new  decisions  which  resulted  from  the  actions  they  had  undertaken. 

Figure  4  proposes  a  construction  of  the  practitioner's  decision-making  process.  As  the  figure 
shows,  decision  making  is  a  dynamic  process  in  which  continual  exploring,  searching,  analyzing, 
constructing,  prioritizing  and  visualizing  activities  may  be  carried  out  simultaneously  or  sequentially.  It  is 
further  proposed  that  these  activities  take  place  within  the  architectural  structure  of  the  interlacing  network 
of  situational  factors. 

Edward  decided  to  shift  his  focus  from  the  farmers  in  his  workshop  to  the  government  personnel 
because  he  was  concerned  about  the  project's  eventual  effectiveness  and  his  client's  satisfaction.  At  this 
decision  point,  client's  needs  and  the  nature  of  the  project  were  the  key  situational  factors  which  were 
immediately  related  (solidly-linked),  while  the  stakeholders  and  resource  availability  were  the  more  distantly 
related  ones  (subtly-linked).  Edward  decided  to  put  the  client's  needs  as  his  first  priority  because  he  had 
grasped  the  essential  relationship  among  these  situational  factors. 

He  had  perceived  that  it  was  the  client,  and  not  CPC,  who  would  provide  most  of  the  ongoing 
support  and  resources  (experts,  capital,  facilities  and  information)  which  would  facilitate  the  project's 
success.  It  was  also  the  client  who  had  a  closer  relationship  with  the  farmers  and  could  diffuse  what  had 
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been  taught.  This  already  solidly  established  relationship,  combined  with  an  effective  diffusion  system, 
would  benefit  the  farmers  and  fulfill  the  government's  policies,  as  well  as  stimulate  repeat  business  for 
CPC. 

In  Edward's  decision-making  process,  he  continually  sought  out  new  information  (e.g.,  through 
meetings  with  the  client  and  reading  suggestions  from  the  post-training  evaluation  questionnaires).  He  also 
searched  for  pattern  matches  by  reviewing  his  previous  experiences,  and  he  aoked  himself  who  his  real  client 
was.  The  updated  knowledge  not  only  helped  him  clarify  the  nature  of  the  problem,  it  also  deepened  his 
understanding  of  the  interrelationships  of  the  situational  factors  involved. 
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Figure  4:  Model  of  the  Corporate  ID  Practitioner's  Decision-Making  Process 

(from  research  on  "The  impact  of  situational  factors  on  the  corporate  instructional  development  practitioner's 
decision  making,"  Liang.  C.  1994) 


In  Mary's  case,  her  team  encountered  a  crisis  because  of  its  loss  of  financial  support.  Its  survival 
depended  on  whether  it  could  successfully  generate  support  from  both  its  organization  (CPC)  and  its  clients 
(Taiwan  businesses).  At  this  decision  point,  organizational  influences  and  client's  needs  became  the  critical 
situational  factors.  After  mentally  analyzing  the  problem,  Mary  began  to  gather  more  information  and 
build  support  by  discussing  the  situation  and  possible  actions  with  hei  team  members  and  top  mar.agernent. 

These  discussions  brought  out  numerous  innovative  ideas  for  creating  market  niches.  The 
consequences  of  each  idea  were  then  examined  against  the  criteria  of  profitability.  Mary  then  gradually 


moved  from  exploration  of  the  problem  to  a  commitment  to  the  situation.  Subsequently,  the  Quality  Club 
was  formed  both  to  secure  a  basic  source  of  revenue  and  to  create  an  information  network  to  the  business 
community.  The  Planning  Professional  seminar  was  carried  out  to  enhance  her  team  members'  professional 
capabilities,  and  to  equip  them  for  the  long  run. 

In  taking  on  the  Informatics  project,  Tony  followed  his  client's  intention  of  improving  the 
contracts  process  only.  He  mentally  and  operationally  went  through  the  same  reflexive  deusion-making 
process  as  Mary.  Likewise,  Emilie's  decisions  on  reforming  the  Knowseed's  incentive  system  and  her 
strategies  for  moving  beyond  the  role  of  an  ID  practit  ioner  shar,;d  the  same  characteristics. 

The  findings  of  this  inquiry  answer  the  original  research  questions.  They  also  suggest  answers  to 
the  following  additional  questions: 

•  How  does  the  experienced  ID  practitioner  arrive     lecisions  that  a  choice  or      action  is  feasible 

in  an  identified  situation? 

•  What  strategies  do  the  experienced  practitioner  use  in  diagnosing  a  situation?  and 

•  How  does  the  experienced  practitioner's  choice  of  action  fit  within  the  broader   framework  of 

his/her  reasoning  about  the  situation? 

Corporate  ID  Practice 

Throughout  this  inquiry,  the  findings  have  consistently  shown  support  for  the  perspective  of  ID  as 
an  eclectic  discipline.  Whether  or  not  ID  activities  refer  to  a  specific  process,  to  a  set  of  criteria,  or  to  a 
social  intervention,  appropriateness  emerges  as  the  key.  Effective  ID  involves  a  subtle  and  sensitive  blend 
of  all  the  above  perspectives,  which  varies  according  to  the  needs  of  the  project,  the  context  in  which  the 
project  takes  place,  and  the  people  involved  in  that  project,  especially  the  client.  Experienced  practitioners 
are  flexible  enough  to  adjust  their  mindsets  and  approaches  according  to  the  different  needs  and  situations 
which  evolve. 

This  inquiry  also  supports  the  contention  that  ID  is  an  interdisciplinary  profession.  In  order  to 
perform  effectively,  experienced  ID  practitioners  transcend  the  boundaries  of  their  professional  discipline. 
This  transcendence  enables  them  to  make  significant  contributions  to  the  clients  they  serve,  and  to  the 
society  in  whi  '..  they  reside.  ID  as  an  interdisciplinary  profession  becomes  even  more  appropriate  in  the 
emerging  "boundaryless"  corporate  environment. 

This  inquiry  shows  its  concurrence  with  and  bias  toward  recent  developments  in  ID  which 
acknowledge  diverse  social,  political  and  cultural  influences  on  professional  ID  practice.  Experienced 
practitioners  are  sensitive  to  any  newly  emerging  situations  and  willingly  adopt,  a  more  flexible  approach  to 
their  projects.  They  emphasize  action-taking  and  action-learning,  and  offer  both  educational  and  non- 
educational  interventions.  These  practitioners  lead  high-performance  teams  competently  and  demonstrate  the 
ability  to  execute  many  assignments  concurrently.  They  satisfy  their  clients,  accommodate  the 
stakeholders,  and  they  promote  their  organization's  interests  and  their  own  professional  growth  at  the  same 
time. 

Based  on  the  findings  of  this  inquiry,  a  conceptual  framework  for  .urporate  ID  practice  was  created. 
This  framework  is  descriptive  in  nature  and  refers  to  no  specific  technical  procedure.  It  operates  within  the 
architectural  structure  of  the  interlacing  network  of  situational  factors.  As  the  conceptual  framework 
illustrates  in  Figure  5,  in  a  particular  cultural  context,  the  process  of  a  corporate  ID  project  brings  the 
practitioner  and  client  systems  together. 

Three  major  activities  take  place:  diagnosis,  action  and  evaluation.  These  are  carried  out  either 
sequentially  or  concurrently,  as  diverse  situations  continually  evolve.  The  diagnosis  activity  includes  needs 
and  performance  capacities  analyses.  Action  represents  the  implementation  of  resolutions.  Evaluation 
consists  of  performance  evaluation  and  revision.  The  client,  practitioner,  and  relevant  stakeholders  are  at  the 
confluence  of  the  three  activities.  Endeavors  devoted  to  creating  values  for  these  participants  are  situated  in 
this  central  realm.  The  interface  between  activities  and  participants,  the  three  overlapping  segments  of  any 
two  activities,  are  areas  full  of  continual  communication,  negotiation  and  interaction. 
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Figure  5:  Conceptual  Framework  of  Corporate  Instructional  Development 

(from  research  on  "The  impact  of  situational  factors  on  the  corporate  instructional  development  practitioner's 
decision  making,"  Liang,  C.,  1994) 


Cultural  context  in  the  conceptual  framework  represents  a  complex  network  in  which  traditional 
values  are  interwoven  with  contemporary  values  and  issues.  Cultural  context  acts  as  a  filter  to  help  the 
project  participants  preserve  things  that  fit,  but  screen  out  those  that  do  not.  It  directs  the  participants' 
behaviors  based  on  their  belief  systems. 

An  ongoing  situation  analysis  and  synthesis  is  carried  out  throughout  the  project  to  help  the 
practitioner  make  decisions  and  to  guarantee  performance  quality.  The  practitioner  grasps  the  critical 
situational  factors  and  their  interrelationships,  and  then  takes  strategic  action(s)  to  optimize  the  situation 
and  to  achieve  full  utility  of  the  resources  available  to  him/her. 

The  diagnosis  activity  is  aimed  at  exploring  both  the  client's  business  needs  and  the  practitioner's 
performance  capacities.  Negotiation  permeates  this  activity.  The  continual  dialectic  between  the 
practitioner  and  the  other  project  participants  clarifies  ill-defined  project  goals,  or  restructures  or  reinforces 
the  established  ones.  The  core  agenda  is  to  reach  a  jointly  satisfactory  match  of  needs  and  capacities. 
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While  on  the  client  side,  needs  analysis  has  been  explored  thoroughly  in  the  ID  literature,  on  the 
practitioner  side,  performance  capacity  analysis  has  received  little  attention  and  has  seldom  been  addressed. 
An  analysis  of  performance  capacities  should  include  a  precise  scrutiny  of  the  available  means  (e.g., 
professional  competencies  and  resources)  necessary  to  implement  the  project.  And,  it  should  be  done  from 
the  practitioner's,  the  client's  and  the  stakeholders'  perspectives.  It  should  encompass  an  examination  of  the 
practitioner's  values,  mindset,  and  professional  ethics. 

The  action  activity  implements  the  mutually  agreed  on  resolutions  developed  from  the  diagnosis 
activity.  If  the  problems  identified  are  knowledge  and/or  skill  deficiencies,  then  educational  interventions 
such  as  training  or  job  aids  are  proposed  and  carried  out.  Relevant  techniques  are  taken  into  account,  e.g., 
task,  analysis,  instructional  strategies,  media  selection,  design  and  production  of  materials.  But,  if  the 
problems  are  related  to  performance  discrepancies,  then  non-educational  interventions  are  suggested  and 
implemented,  e.g.,  motivating,  promoting  communication,  supplying  timely  feedback,  redesigning  jobs,  or 
reforming  the  organizational  structure. 

All  the  interventions  employed  to  resolve  these  problems  are  situation-specific,  and  are  directed 
toward  purposeful  action.  In  order  to  optimize  the  effect  of  the  solutions,  the  interventions  must  be  linked 
to  the  cliert's  system  and  become  an  organic  part  of  their  daily  operations. 

The  evaluation  activity  encompasses  both  ongoing  performance  evaluation  and  revision.  This 
activity  is  an  effort  to  create  a  joint  agreement  about  the  project's  worth,  similarly  proposed  by  Coleman,  et 
al„  (in  press).  The  critical  concerns  of  evaluation  are:  the  practitioner's  ethical  reflections,  the  resolutions 
of  the  consulting  subject,  allocation  of  project  resources,  the  client's  agenda,  the  social  dynamics  of  the 
client  system,  the  impact  on  both  the  practitioner's  and  client  system,  and  potential  for  the  repeat  business. 

A  corporate  ID  project  may  include  all  of  the  above  three  activities  (diagnosis,  action  and 
evaluation)  or  any  combinations  of  these.  The  project  may  start  from,  or  end  at,  any  activity.  Either 
starting  or  ending  a  project  is  enacted  at  the  overlapping  areas  of  any  two  of  these  three  activities. 

These  three  overlapping  areas,  the  interface  between  the  participants  and  the  activities,  are  filled 
with  constant  interactions  among  the  project  participants.  These  are  the  areas  rich  with  different 
perceptions,  experiences,  values,  and  cultures,  i.e.,  areas  crowded  with  "human  obstacles"  as  termed  by  Arnn 
&  Strickland  (1975).  It  is  in  these  interfacing  areas  that  the  project  participants  identify  and  overcome 
various  human  obstacles.  Tasks  conducted  in  these  areas  may  include  communicating,  negotiating, 
compromising,  reviewing,  contracting,  and  managing  or  terminating  the  project. 

All  efforts  engaged  in  the  project  are  aimed  at  creating  values  to  the  client,  to  the  relevant 
stakeholders,  and  to  the  practitioner.  The  practitioner,  while  maintaining  standards  of  professional  ethics, 
exercises  his/her  competitive  advantages  and  strives  for  the  project's  success.  Success  emphasizes  win-win 
outcomes— not  only  economic  objectives,  but  the  achievement  of  "social  satisfaction"— delighted  clients, 
satisfied  stakeholders,  and  professionally  evolving  practitioners. 

Future  Directions 

This  study  has  opened  many  areas  worthy  of  further  exploration.  These  areas  are  synthesized  under 
two  major  themes:  ( 1 )  corporate  ID  practice  in  general,  and  (2)  the  experienced  practitioner's  decision- 
making process  in  more  specific  detail. 

Corporate  ID  Practice 

Future  research  on  corporate  ID  practice  might  focus  on  this  topic  in  different  contexts  (e.g., 
private  corporations,  academic  environments  or  other  cultural  contexts)  and/or  with  different  methodologies 
(either  under  a  naturalistic  or  positivist  paradigm).  Far  more  study  is  needed  on  the  concepts  identified  in 
this  inquiry  such  as  the  situational  factors  and  conceptual  frameworks.  Following  is  a  partial  list  of 
suggestions  which  promise  to  provide  fruitful  results. 


Situational  Factors 

•  the  critical  elements  in  each  situational  factor,  and 

•  the  effects  of  these  critical  elements  on  the  practitioner's  decision  making. 
Conceptual  ID  Frameworks 

•  the  process  models  generated  from  the  emerging  frameworks, 

•  the  prescriptive  or  descriptive  guidelines  developed  from  these  emerging  frameworks  or  process 

models, 

•  the  process  of  situation  analysis  and  synthesis, 

•  the  criteria  for,  and  process  of,  performance  capacity  analysis, 

•  the  effect  of  relationships  among  different  types  of  clients  and  stakeholders  on  the 

practitioner's  decision  making,  and 

•  the  effects  of  different  business  strategies  on  the  practitioner's  decision  making. 

Decision-Making  Process 

Given  the  results  of  the  present  study,  several  questions  regarding  the  practitioner's  decision- 
making process  remain  unanswered.  Themes  recommended  for  further  study  to  extend  the  research  begun 
here  are:  (a)  the  individual  practitioner  and  project  team,  and  (b)  corporate  ID  training. 

Individual  Practitioner  and  Team  Issues 

•  the  types  of  decision-making  errors  made  in  ID  project, 

•  the  strategies  for  handling  these  decision-making  errors, 

•  the  effects  of  various  stressors  (e.g.,  time  pressure)  on  decision  making, 

•  the  strategies  for  overcoming  project  problems  with  incomplete  and/or  inaccurate 

information, 

•  the  effects  of  increased  hands-on  experience  on  decision  making, 

•  the  effects  of  increasing  knowledge  of  the  subject  matter  on  decision  making, 

•  the  major  issues  in  corporate  ID  team  decision  making,  and 

•  the  process  and  strategies  of  team  decision  making. 
Corporate  ID  Training  Issues 

•  the  curriculum  for  training  ID  practitioners  in  situational  assessment  and  decision  making  skills, 

and 

•  the  instructional  methods  or  strategies  for  training  ID  practitioners  in  situational 

assessment  and  decision  making  skills. 

The  answers  to  these  questions  promise  to  significantly  improve  our  understanding  of  corporate  ID 
decision  making,  as  well  as  provide  an  intellectual  foundation  for  advanced  studies  of  professional  ID 
practice  in  the  corporate  setting. 
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Abstract 


This  exploratory  study  was  designed  to  introduce  hypertext  as  a  component  of  classroom 
instruction  in  a  situation  similar  to  that  which  might  prevail  in  a  public  school  setting.  Linear  and  non- 
linear treatments  of  the  same  text  content  were  employed  as  an  information  source  for  a  supplemental 
learning  task  prescribed  by  a  teacher  as  part  of  classroom  instruction.  The  participants  were  elementary 
school  children  grouped  by  experience  with  video  game  technology  in  order  to  study  the  influence  of  that 
experience  on  their  performance  and  satisfaction  with  the  two  hypertext  versions.  Performance  was  assessed 
by  a  student  activity  sheet,  a  delayed  post-measure,  and  an  attitude  survey.  There  were  no  significant  main 
effects  or  interactions  found  for  any  of  the  dependent  variables.  A  post  hoc  exploratory  analysis  revealed  a 
significant  main  effect  for  gender  on  attitude  toward  lesson  organization.  Males  found  the  lesson,  regardless 
of  treatment,  more  clearly  organized  than  did  females. 


Linear  and  Non-linear  Hypertext 
in  Elementary  School  Classroom  Instruction 

Hypertext  is  a  form  of  text  made  non-linear  through  integrating  a  system  of  nodes  and  links.  Each 
node  of  information  (often  a  paragraph)  may  be  linked  to  any  number  of  other  nodes,  which  are  themselves 
linked  to  yet  other  nodes.  The  nodes  of  text  are  interconnected  through  this  potentially  complex  series  of 
links,  which  can  be  organized  to  convey  a  variety  of  conceptual  relationships  (e.g.  hierarchical,  spatial, 
chronological,  semantic)  alone  or  in  combination.  The  nature  of  these  linking  relationships  may  or  may 
not  be  labeled  or  defined  for  the  user. 

The  hypertext  user  moves  from  node  to  node  through  the  linked  web  of  textual  information  by 
selecting  a  link  from  among  those  available  at  each  node.  The  resultant  patn  through  the  text  cannot  be 
predicted  with  certainty  by  the  hypertext  author,  and  may  vary  greatly  from  user  to  user.  The  effect  of  this 
random  access  within  the  hypertext  document  is  "to  overcome  the  strictly  sequential  medium  of  print  and 
paper"  (Marchionini,  1988,  p.  8). 

The  hypertext  concept  of  allowing  unlimited  quantities  of  knowledge  to  be  stored,  organized,  and 
retrieved  by  the  user  in  any  organizational  scheme  desired,  is  intuitively  appealing,  but  has  been  difficult  to 
define.  The  fact  that  the  technology  which  allows  the  creation  of  hypertext  is  emerging  and  changing  even 
as  hypertext  is  being  invented  and  refined  makes  defining  this  medium  even  more  difficult,  nevertheless, 
numerous  definitions  have  been  proposed.  Aust  and  colleagues  (1993)  have  done  as  well  as  anyone: 
"Hypertext  is  an  array  of  emerging  technologies  for  accessing,  organizing,  and  relating  electronic-based 
information."  As  the  technology  developed  to  allow  visuals,  sound,  and  video  to  be  stored  and  arranged  in 
conjunction  with  text,  the  term  hypermedia  was  coined.  Ramaiah  (1992),  in  a  summary  of  hypertext 
issues,  offers  this  definition:  "When  a  multimedia  program  is  developed  in  a  hypertext  environment,  the 
resulting  product  is  called  hypermedia." 

This  definition,  of  course,  raises  the  question  of  what  constitutes  a  hypertext  environment.  Mark 
Horncy  (1991 )  has  specified  defining  characteristics  of  a  hypertext  environment:  (a)  information  stored  in 
freestanding  nodes,  (b)  nodes  connected  by  associative  links,  (c)  active  engagement  required  of  users  and 
authors,  (d)  the  system  not  dependent  on  any  particular  informational  structure  or  medium  of  presentation. 

Although  not  a  new  concept,  hypertext  has  been  readily  available  for  instructional  use  only  since 
the  recent  introduction  of  HyperCard  and  similar  software  for  personal  computers.  This  availability  has 
generated  interest  in  potential  instructional  applications  of  hypertext.  Interest  has  resulted  in  the  appearance 
of  numerous  theoretical,  conceptual,  and  speculative  essays  on  hypertext.  Reviewers  of  this  literature  (Tsai, 
1988;  Heller,  1990;  Ambrose,  1991;  Park,  1991)  have  identified  characteristics  of  hypertext  such  as  non- 
linear structure,  linking  ability,  and  ability  to  adapt  to  or  support  a  learner's  associative  thinking  processes. 

In  addition  to  identifying  characteristics,  the  reviewers  of  hypertext  literature  often  list  problems 
associated  with  learner  use  of  hypertext.  Jonassen  (1988)  provides  atypical  list  of  items:  (a)  navigation 
through  hypertext,  (b)  selecting  an  appropriate  user  access  point  within  a  hypertext  document,  (c) 
integration  by  learners  into  their  own  knowledge  structure  what  has  been  learned  in  the  hypertext,  (d) 
cognitive  overload  caused  by  the  richness  of  non-linear  representation. 
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Research  and  Theory 


One  area  discussed  by  theorists  but  not  directly  addressed  by  research  is  die  issue  of  hypertext  as  a 
new  discourse  form  and  the  related  issue  of  developing  a  rhetoric  for  that  form.  Norton  (1992,  p.  42) 
maintains  that  "Hypertext/hypermedia  as  discourse  suggests  that  knowledge  can  be  derived  not  only  from 
hierarchical,  analytical  relations  of  superordinance  and  subordinance,  but  also  as  complex  networks  which 
transcend  narrow,  disciplinary  boundaries."  According  to  Norton  (p.  40),  this  leads  to  "...  a  shift  in  both 
the  process  and  outcomes  of  knowing."  Bevilacqua  (1989,  p.  162)  says,  "Over  time  and  use,  hypertext  will 
probably  change  our  way  of  thinking.  . .  ."  Slatin  (1990,  p.  871),  in  his  discussion  of  a  possible  rhetoric 
for  hypertext,  points  out  that  presently,  "The  assumption  that  reading  is  a  sequential  and  continuous  process 
is  the  foundation  on  which  everything  else  rests."  He  goes  on  to  identify  three  classes  of  hypertext  readers: 
browsers,  users,  and  co-authors.  "The  relationship  between  these  three  classes  can  be  fuzzy  and  therefore 
difficult  to  manage"  (p.  875). 

The  published  literature  on  hypertext  contains  relatively  little  experimental  research.  Jonassen's 
(1988)  assertion  that,  "Hypertext  design  is  theory-rich  and  research-poor,"  remains  accurate  today  In  the 
hypertext  research  literature,  many  studies  deal  with  large  hypertext  database  systems,  which  Rada  (199 1) 
has  termed  "large  volume"  or  "macrotext."  These  studies  address  issues  such  as  the  use  of  browsing  tools 
to  aid  in  navigating  through  large  hypertext  database  systems  containing  numerous  documents  (Nielsen, 
1990).  "Small  hypertext",  (microtext),  consisting  of  single  documents  or  lessons,  can  now  be  authored  and 
used  on  inexpensive  personal  computers  in  classrooms  and  other  instructional  settings.  At  this  time,  there 
is  relatively  little  research  to  guide  the  design  or  effective  utilization  of  this  instructional  hypertext. 

Of  the  studies  exploring  instructional  applications  of  hypertext,  only  a  few  address  the 
implications  of  its  characteristic  of  non-linearity  in  an  instructional  setting.  Yet,  in  the  literature,  the 
hypertext  problems  most  frequently  cited  relate  directly  to  the  non-linear  nature  of  the  medium.  A  variety 
of  terminology  is  used  to  describe  this  problem  area,  including:  cognitive  overload,  getting  lost  in 
hyperspace,  unmotivated  rambling,  and  disorientation.  The  theo'  ists  suggest  that  hypertext  may  be  too 
unstructured  and  confusing  for  learners  to  use  efficiently  and  effectively. 

The  experimental  results  with  instructional  hypertext  have  been  limited  and  mixed.  McGrath 
(1992),  Way  (1992),  Lin  (1991),  Lanza  and  Roselli  (1992),  and  Harris  (1990)  all  tested  students'  mastery  of 
lesson  content  after  completing  alternate  versions  of  the  same  lesson.  In  each  study,  a  non-linear  hypertext 
version  was  compared  to  one  or  more  linear  presentations  of  the  same  text. 

McGrath  (1992)  compared  four  treatment  types:  hypertext,  CAI,  paper  and  a  "NoMenu"  linear 
computerized  text.  College  undergraduates  were  assigned  to  high  and  low  spatial  skills  groups  and  received 
instruction  in  calculating  surface  area  and  volume.  While  the  study  focused  primarily  on  the  issues  of 
learner  control  and  student  misconceptions,  the  treatments  varied  as  well  in  linearity.  Subjects  in  the  paper 
treatment  group  scored  highest  on  the  post-test  problems,  followed  by  hypertext,  CAI  and  NoMenu.  The 
hypertext  group  took  the  least  amount  of  time  to  complete  the  lesson. 

Way  (1992)  compared  three  lesson  treatments:  non-linear  hypertext,  linear  CAI,  and  linear  CAI 
with  repetition  option.  The  subjects  were  college  undergraduates.  The  lesson  dealt  with  basic 
psychological  terms.  While  there  were  no  significant  differences  in  post-test  achievement  scores,  the 
hypertext  group  took  significantly  longer  to  complete  the  lesson.  However,  a  post-hoc  examination  of 
learners'  paths  through  the  hypertext  showed  that  those  in  the  hypertext  treatment  group  who  selected  a  non- 
linear path  through  the  lesson,  completed  it  in  the  same  amount  of  time  while  still  scoring  as  well  as 
students  in  the  other  two  treatments. 

Lin  (1991)  compared  linear  and  non-linear  computer-based  word  processing  tutorials  with  college- 
age  subjects.  Two  of  the  three  treatment  groups  were  given  the  same  hypertext  instruction.  One  of  these 
hypertext  groups  received  ten  minutes  of  training  and  practice  on  the  features  of  the  hypertext  just  before  the 
lrsson.  A  third  treatment  group  used  a  linear  tutorial.  The  hypertext  group  with  training  and  practice 
scored  significantly  higher  on  both  the  immediate  and  the  delayed  post-tests. 

Lanza  and  Roselli  (1991)  compared  a  linear  CAI  treatment  with  a  hypertext  in  computer 
programming  instruction  at  the  college  level.  Post-test  results  found  no  significant  advantage  for  either 
treatment,  though  more  hypertext  subjects  found  their  program  "stimulating  and  attractive."  The  authors 
noted  that  while  mean  group  scores  did  not  differ,  the  range  of  scores  in  the  hypertext  group  was  greater 
than  in  the  linear  group. 

Harris  (1990)  compared  computer-based  linear  text  and  hypertext  in  college  computer  literacy 
instruction.  Instruction  consisted  of  a  lesson  on  computer  communication  lasting  about  forty-five  minutes 
and  followed  by  a  post-test.  There  were  no  significant  differences  in  post  test  scores  by  treatment. 
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Achievement  results  by  gender,  and  the  amount  of  time  required  to  complete  the  instruction  also  yielded  no 
significant  results. 

While  the  results  of  these  studies  do  not  demonstrate  superior  achievement  results  for  non-linear 
hypertext  as  an  alternate  delivery  system  for  computer-based  lessons,  they  show  that  in  some  circumstances 
hypertext  can  be  equally,  or  in  some  cases  more,  effective  than  other  systems  for  delivering  textual 
information  to  learners.  However,  the  studies  reveal  no  clear  pattern  of  superior  time  efficiency,  post  test 
improvement,  or  lesson  enjoyment,  for  any  one  lesson  type. 

Implications  of  Hypertext  Literature 

If  the  goal  is  to  identify  effective  uses  of  hypertext  in  instruction,  these  results  suggest  that  it  may 
not  be  not  fruitful  to  approach  hypertext  only  as  an  alternate  delivery  system  for  factual  information  in 
computer-based  lessons.  Revisiting  the  characteristics  of  the  medium,  one  might  suppose  that  hypertext's 
strength  could  lie  in  the  representation  of  conceptual  relationships  among  textual  information,  and  that  its 
use  in  instruction  might  best  involve  modeling  or  representing  those  relationships  for  learners  (Jonassen, 
1988;  Carlson,  1990).  While  some  hypertext  systems  are  being  developed  according  to  the  principles  of 
cognitive  flexibility  theory  to  provide  models  for  use  with  case-based  instruction  in  complex  and  ill- 
structured  domains  (Spiro,  Feltovich,  Jacobson,  and  Coulson,  1991),  this  approach  has  not  been 
investigated  on  a  smaller  scale  within  a  more  clearly  structured  domain. 

Another  potentially  important  issue  not  addressed  in  any  of  the  studies  is  suggested  by  McGrath 
(1992,  p.  529)  in  her  discussion  of  results:  "Apparently  22-year-old  students  are  simply  more  accustomed 
to  learning  from  paper  than  from  computer  regardless  of  degree  of  learner  control.  Students  will  need  more 
practice  using  the  computer  for  learning."  It  may  be  significant  that  the  participants  in  all  four  studies  were 
college-age  or  adult  learners.  It  is  possible  that  the  factor  of  learner  age  (and  its  attendant  prior  experience 
and  mind-set)  could  significantly  affect  a  learner's  approach  to  and  success  with  a  non-linear  medium 
presented  via  computer. 

Norton  (1992)  in  her  essay  on  the  computer  as  discourse,  points  out  that  the  relationship  between 
the  sender  of  a  message,  the  receiver,  and  the  topic  is  culturally  determined.  Observing  how  today's 
elementary  school  age  child,  born  'ato  a  world  of  videocassette  recorders  and  remote  controls,  uses 
television  might  be  helpful  to  those  interested  in  analyzing  and  researching  hypertext.  This  child  may 
readily  browse  through  his  or  her  favorite  tapes,  skipping  passages  and  reviewing  at  will.  Television  for 
this  child  appears  much  less  linear  than  for  passive  television  watchers.  The  same  child  may  also  be 
observed  spending  hours  navigating  the  obscure  levels  of  several  video  games,  searching  for  strategies 
which  allow  non-linear  movement  among  the  "worlds"  and  screens  of  the  games. 

Norton  might  say  that  the  child  has  acquired  a  new  discourse  form  not  generally  practiced  by 
adults;  a  discourse  form  probably  not  anticipated  by  observers  at  the  time  remote  controls  or  video  games 
were  first  introduced.  It  might  be  fruitful  to  speculate  that  a  similar  process  may  occur  with  the  increasing 
availability  of  hypertext.  Perhaps  children  born  into  a  world  in  which  hypertext  is  common  will  adopt  that 
discourse  form  as  readily  as  today's  children  have  adopted  video.  It  has,  however,  been  suggested  that 
younger  learners  would  be  distracted  by  the  nonessential  information  presented  and  cognitive  overhead 
required  in  the  use  of  hypertext  (Heller,  1990). 

Although  there  has  been  some  investigation  of  the  relationship  between  children's  attitudes  toward 
computers  and  their  attitudes  toward  video  games  (Fisher  and  Pulos,  1983),  the  potential  effects  of  their 
video  experience  on  success  with  computer  applications  such  as  hypertext  have  not  been  explored.  It  is 
possible  that  these  video  experiences  have  encouraged  younger  learners  to  develop  some  skills  or  attitudes 
which  allow  them  to  work  more  effectively  than  today's  adults  with  non-linear  hypertext  in  instructional 
settings. 

Study  Design  Rationale 

Influence  of  Prior  Research 

Several  characteristics  of  the  design  of  this  exploratory  study  were  influenced  by  the  instructional 
hypertext  studies  previously  cited.  These  factors  include  the  age  of  the  subjects,  the  form  of  the 
instruments  used  to  measure  knowledge  acquisition  and  retention,  and  the  inclusion  of  an  experience  factor 
in  the  experimental  design. 

Testing  instructional  hypertext  with  younger  subjects  appeared  to  be  an  obvious  extension  of  the 
previous  studies,  all  of  which  were  done  with  young  adults.  A  recently  published  instructional  hypertext 
study  (Shin,  Schallert  and  Saveyne,  1994)  has  taken  this  same  approach.  A  further  consideration  in  the 
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decision  to  work  with  younger  learners  was  to  investigate  the  potential  influence  of  children's  video 
technology  experiences  on  their  use  of  a  non-linear  discourse  form.  Using  younger  subjects  allowed  the 
introduction  of  video  game  expertise  as  an  experience  factor.  In  an  article  published  after  this  study  was 
conducted,  Kahn  and  Landow  (1993)  have  also  argued  for  the  importance  of  considering  prior  experience  as 
an  important  factor  in  learners'  success  with  hypertext. 

Post  tests  employed  in  previous  studies  measured  retention  of  factual  knowledge,  or  success  in 
applying  the  formula  modeled  in  instruction,  consisting  of  various  formats  of  computerized  lessons.  As 
Spiro  et  al.  (1991)  suggest,  presenting  items  of  verbal  information  or  modeling  a  step-by-step  process  may 
not  be  the  most  appropriate  instructional  applications  for  a  non-linear  medium  like  hypertext. 
Consequently,  retention  or  application  of  this  sort  of  information  may  not  be  the  best  measure  of  the 
effectiveness  of  the  medium.  In  addition,  such  measures  in  previous  studies  had  delivered  no  clear  pattern  of 
results. 

For  this  study,  a  student  activity  sheet  was  chosen,  in  an  attempt  to  measure,  not  the  retention  of 
facts,  but  the  quantity  of  information  accessed  in  the  process  of  solving  a  teacher-assigned  problem.  It  was 
designed  to  allow  comparison  of  how  effectively  students  using  the  linear  and  non-linear  versions  of  a  text 
accessed  and  used  the  required  information.  The  delayed  post-test  instrument  selected  was  modeled  on  a 
graphic  organizer  (Pehrsson  and  Robinson,  1985).  Its  selection  and  design  were  influenced  by  the 
theoretical  discussions  of  potential  relationships  between  hypertext  structures  and  learners'  knowledge 
structures.  It  was  designed  to  assess,  not  only  the  quantity  of  factual  information  retained,  but  also  the 
effect  of  a  hypertext  structure  on  learners'  perceptions  of  relationships  between  those  facts. 
Influence  of  School  Experience 

Another  major  influence  on  the  design  of  this  exploratory  study  was  the  author's  experience  as  a 
media  technology  specialist  at  the  elementary  school  level.  Observation  of  elementary  school  classroom 
practices,  and  experiences  with  computer  availability  and  utilization  in  elementary  schools,  influenced 
decisions  about  the  curriculum  content,  the  instructional  role  of  the  hypertext  materials,  the  assessment 
instruments,  and  the  scheduling  of  the  experimental  treatments. 

The  hypertext  treatment  in  other  studies  has  been  a  teacher-independent  computerized  lesson 
matched  with  other  CAI  or  paper-based  treatments  of  the  same  lesson.  This  type  of  hypertext  application 
may  be  unrealistic  for  public  schools,  given  the  limited  availability  of  hardware  and  the  labor-intensive 
instructional  delivery  commonly  found  in  the  school  setting.  In  an  attempt  to  model  common  elementary 
school  classroom  practice,  the  hypertext  in  this  study  was  introduced  as  one  of  the  several  information 
sources  being  prescribed  as  part  of  teacher-directed  instruction. 

The  content  of  the  hypertext  lessons  employed  in  previous  studies  has  consisted  primarily  of 
verbal  information  (psychological  terms,  computer  terminology)  and  application  (using  a  mathematical 
formula,  introductory  word  processor  operations).  The  text  content  used  in  this  study  consisted  of 
interconnected  facts  comprising  a  body  of  knowledge.  The  assigned  problem  required  the  student  to  locate 
specific  factual  information  from  within  that  body  of  knowledge  and  to  employ  those  required  facts  in  a  task 
(worksheet)  involving  reporting,  restructuring,  or  classifying  the  information.  The  design  was  an  attempt 
to  shift,  as  Spiro  et  al.  (1991)  suggest,  from  an  orientation  "that  emphasizes  the  retrieval  from  memory  of 
intact  preexisting  knowledge  to  an  alternative  constructivist  stance  which  stresses  the  flexible  reassembly  of 
preexisting  knowledge  to  adaptively  fit  the  needs  of  a  new  situation"  (p.  25).  This  design  was  also  an 
attempt  to  employ  the  hypertext  as  a  supplemental  information  source  supporting  a  learning  task  assigned 
by  a  teacher  as  part  of  classroom  instruction.  The  learners  were  required  to  function  as  "users"  and/or 
"browsers"  (Slatin,  p.  871)  as  they  worked  on  teacher-directed  tasks  which  required  information  contained  in 
the  hypertext.  The  intent  of  the  design  was  to  create  a  classroom  environment  in  which  the  hypertext  was 
used  as  one  of  several  textual  tools  available  to  the  instructor  and  the  learners  as  they  progressed  through  a 
unit  or  lesson  -  a  potential  instructional  role  for  hypertext  which  could  be  a  good  match  to  its  intrinsic 
characteristics. 

Since  in  many  school  settings,  access  to  computer  labs  is  limited,  the  use  of  hypertext  in 
computer-based  lessons  is  also  likely  to  be  limited.  Teachers  are  more  likely  to  have  access  to  one  or  two 
computers  within  the  classroom.  The  use  of  hypertext  in  this  study  in  a  supplemental  assignment  which 
could  be  done  independently  by  students  throughout  the  school  day,  was  an  attempt  to  create  a  design  which 
could  study  hypertext  use  in  a  setting  similar  to  that  which  might  actually  exist  in  many  elementary  school 
classrooms. 
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Materials 


Hypertext 

The  content  of  the  hypertext  used  in  this  study,  the  geological  regions  of  Minnesota  and  their 
products,  is  related  to,  but  does  not  duplicate,  the  content  of  a  required  fourth  grade  social  studies  unit. 
Prior  to  the  study,  the  hypertext  material  was  tested  with  two  fourth  grade  classes  in  another  school.  The 
vocabulary,  reading  level,  and  structure  of  the  presentation  were  judged  by  fourth  grade  teachers  as  being 
appropriate  for  that  grade  level. 

In  the  non-linear  version,  the  hypertext  web  radiates  from  numerous  graphic  choice  screens.  The 
content  consists  of  information  on  each  of  four  geologic  regions.  The  content  within  each  region  is 
organized  into  four  sub-topics:  land,  products,  plants,  and  the  past.  Within  each  sub-topic  the  content  is 
divided  into  an  information  section  and  an  activities  section.  On  entering  the  non-linear  version,  the  student 
selects  a  region  from  the  choice  screen.  Whenever  a  region  is  selected,  the  four  sub-topic  choices  are  then 
presented  for  selection.  The  choice  between  the  information  and  activities  sections  is  similarly  presented  on 
entering  each  subtopic.  The  structure  provides  some  guidance  for  topic  selection,  in  that  as  each  sub-topic 
on  a  region  is  completed,  the  student  is  returned  to  that  region's  choice  screen.  However,  free  access  to  the 
entire  web  is  available  from  every  screen  through  a  consistent  "quit"  button.  Pressing  this  button  at  any 
point  returns  the  user  to  a  summary  choice  screen  which  graphically  displays  the  four  regions  and  the  four 
sub-topics  within  each  region,  with  check  marks  by  each  which  the  student  has  previously  selected.  A  click 
on  any  of  these  icons  moves  the  student  directly  to  that  region  and  topic.  This  allows  students  at  any  point 
to  select  screens  viewed  earlier  in  the  lesson,  move  on  to  new  information,  return  to  previous  choice  points, 
or  quit  the  lesson. 

The  non-linear  hypertext  was  modified  to  create  a  linear  version  which  presents  all  the  screens  of 
the  non-linear  version,  but  in  a  predetermined  sequence.  In  the  linear  version,  the  learner  has  at  all  times 
the  option  of  continuing  through  the  pre-determined  sequence  of  screens,  reversing  the  sequence  one  screen 
at  a  time,  or  quitting.  There  are  no  choice  screens  available,  except  a  warning  screen  to  determine  if  a 
student  who  selects  "quit"  really  wishes  to  do  so. 

Both  versions  were  constructed  to  allow  one  thirty  minute  work  session  per  user.  When  the  time 
expires,  the  student  is  forced  to  quit  the  lesson.  This  feature  was  included  to  allow  comparison  of  the 
amount  information  located  and  reported  on  the  worksheet  by  users  of  linear  and  non-linear  versions  within 
the  same  amount  of  work  time. 
Activity  Sheet 

The  format  of  the  activity  sheet  completed  by  students  during  the  thirty  minute  hypertext  session 
was  similar  to  that  of  materials  used  with  the  required  social  studies  materials.  Content  of  the  items  was 
based  on  the  hypertext.  Some  completion  items  (e.g.  Name  the  main  industry  of  the  Superior  Upland  area 

 .)  required  locating  in  the  text  a  single  fact  and  reporting  it.  Other,  more  complex  items,  required 

locating  facts  from  several  areas  of  the  text  and  combining  them  in  a  single  answer.  For  example,  a 
partially  labeled  Venn  diagram  (a  format  familiar  to  students  from  previous  social  studies  lessons)  required 
recording  some  of  the  farm  products  of  each  of  two  geographic  areas,  and  those  which  arc  common  to  both 
areas.  Several  questions  made  use  of  graphic  maps  similar  to  those  located  throughout  the  text  to  record 
information  found  in  one  or  more  areas  of  the  text. 

The  activity  sheet  was  developed  by  the  researcher  after  reviewing  a  number  of  similar  sheets  used 
in  the  fourth  grade  social  studies  curriculum.  It  was  then  reviewed  and  revised  by  several  fourth  grade 
teachers  who  judged  the  reading  level  and  difficulty  of  the  items  as  appropriate  for  fourth  grade  students. 
Next  it  was  tested  by  observing  individual  fifth  grade  students  complete  it  using  both  the  linear  and  non- 
linear hypertext  versions,  and  by  interviewing  those  students  after  they  had  completed  it.  From  these 
observations  and  interviews,  wording  was  adjusted  to  eliminate  confusing  items,  the  items  were  ordered  so 
that  information  needed  for  an  answer  might  be  found  in  any  part  of  either  hypertext  version,  and  the  length 
of  the  activity  sheet  was  adjusted  to  what  was  estimated  to  take  an  efficient  student  slightly  longer  than 
thirty  minutes  to  complete.  Finally,  a  scoring  system  was  developed  and  tested  in  conjunction  with  a  pilot 
run  of  the  study  using  a  smaller  (N=51 )  sample  of  fourth  grade  students. 
Delayed  Post  Measure 

As  mentioned  earlier,  the  post  measure  was  intended  to  measure  retention  of  factual  information, 
and  to  assess  any  affect  of  the  hypertext's  structure  (linear  or  non-linear)  on  the  student's  awareness  of  the 
relationships  between  those  facts.  Several  possible  test  formats  were  considered,  including  the  ordered  tree 
technique  (Naveh-Benjamin,  et  al,  1986),  word  associations  or  graphing  techniques  (Shavelson,  1974),  and 
pattern  notes  (Jonassen,  1987).  None  of  these  seemed  appropriate  for  an  exploratory  study  with  elementary 
school  age  subjects. 


Eventually,  a  format  modeled  on  a  graphic  organizer  (Pehrsson  and  Robinson,  1985)  was  selected 
as  most  appropriate  for  use  by  this  age  group  in  a  classroom  setting,  efficient  to  administer  and  score,  and 
offering  the  possibility  of  making  some  inference  about  the  effec's  of  the  hypertext  structure  on  students' 
perceptions  of  relationships  between  individual  facts.  Several  designs  were  then  reviewed  by  the  consulting 
teachers.  The  one  selected  was  similar  in  design  to  the  graphic  map  and  choice  screens  used  throughout  the 
hypertext. 

A  single  page  displayed  the  graphic  map  showing  the  four  regions.  Next  to  each  region  a  large 
empty  rounded  rectangle  was  divided  into  four  sections  labeled  with  the  names  of  the  four  sub-topics.  The 
directions  stated,  "On  this  page,  write  or  draw  anything  you  remember  about  the  regions  of  Minnesota." 
Two  examples  of  factual  information  correctly  placed,  one  a  drawing,  one  written,  were  included.  When  the 
test  was  administered,  the  directions  were  repeated  orally  three  times  at  intervals,  along  with  encouragement 
to  think  back  to  the  hypertext  work  time  and  draw  or  write  whatever  came  to  mind.  The  administration  and 
scoring  procedures  were  developed  and  tested  in  conjunction  with  the  cooperating  teachers  at  the  time  of  the 
pilot  run. 

Video  Game  Experience  Assessment  Instrument 

The  instrument  used  to  assess  video  game  experience  and  expertise  was  developed  with  the 
assistance  of  student  volunteers,  the  cooperating  teachers,  and  several  additional  pilot  classrooms.  The  first 
pilot  version  was  constructed  by  intsrviewing  several  highly  experienced  high  school  age  video  game 
players.  It  posed  numerous  factual  questions  about  specific  game  systems  and  strategies.  However,  the 
wide  variety  of  games  and  systems  available  to  students  made  these  questions  too  hard  for  the  panel  of 
elementary  school  video  game  student  experts  who  reviewed  it.  Even  these  acknowledged  sixth  grade  expert 
players  scored  poorly  on  this  initial  version.  Additional  interviews  with  the  younger  users  led  to  the 
development  of  a  second  version  which  combined  survey  questions  about  game  use  (e.g.  "If  you  have  any  of 
these  game  systems  at  home,  circle  the  ones  that  you  have.)  with  a  few  factual  questions  about  the  most 
popular  games  and  systems  of  the  day  (e.g.  "How  many  warp  zones  are  there  in  the  game  Mario 
Brothers?").  This  version  was  then  tested  and  discussed  individually  with  several  fourth  grade  students  of 
varying  video  game  expertise. 

Finally,  a  multi-stage  classroom  testing  process  was  undertaken  which  involved  administering  the 
instrument,  interviewing  students  and  teachers,  and  comparing  the  scores  with  teacher  ratings  and  student 
peer  ratings  of  video  game  expertise.  This  process  was  repeated,  with  minor  revisions  to  the  instrument,  in 
four  classrooms  in  three  different  schools.  In  the  last  of  these  pilot  tests,  the  instrument  achieved  a 
correlation  of  .7  (N=90)  with  the  peer  ratings. 
Attitude  Survey 

The  attitude  survey  was  modeled  on  forms  used  in  previous  studies.  It  contained  twelve  Likert 
scale  questions  designed  to  measure  two  issues:  lesson  enjoyment  (e.g.  "I  would  like  more  lessons  like  this 
one."),  and  lesson  organization  (e.g.  'The  computer  lesson  was  confusing.").  The  sub-scales  for  these  two 
issues  contained  six  items  each.  Of  these,  three  were  worded  positively,  and  three  negatively.  Responses 
were  consistently  labeled  Strongly  Agree  -  Strongly  Disagree,  and  scoring  (1-5)  was  reversed  when 
appropriate  to  the  wording  of  the  question.  The  six  items  for  each  sub-scale  were  combined  and  compiled  in 
random  order  on  a  single  form  for  the  final  twelve  item  test. 

The  Study 


Research  Questions  and  Design 

The  primary  questions  addressed  by  the  study  were  three:  Which  text  structure,  linear  or  non- 
linear, is  used  more  effectively  as  a  source  of  information  in  an  assigned  problem-solving  task?  Does  the 
structure  of  the  hypertext,  linear  or  non-linear,  affect  retention  of  facts  or  perception  of  the  relationships 
between  facts?  Are  frequent  users  of  video  game  technology  more  successful  than  infrequent  game  users 
when  working  with  non-linear  hypertext?  Other  questions  considered  were:  With  which  treatment,  linear  or 
non-linear,  are  participants  most  satisfied?  Do  frequent  and  infrequent  video  game  users  vary  in  their 
preference  for  treatment  type?  Is  preference  for  a  particular  treatment  related  to  higher  scores  on  the  assigned 
task? 

The  experiment  was  of  a  2x2  factorial  design.  The  two  treatment  levels  were  Linear  and  Non- 
linear Hypertext.  The  two  experience  factors  were  High  and  Low  Video  Game  Experience  The  three 
dependent  variables  were  the  number  of  items  completed  correctly  on  the  activity  sheet,  number  of  correctly 
placed  items  on  the  delayed  post  measure,  and  the  Likert  ratings  of  lesson  enjoyment  and  lesson 
organization 
Participants 
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The  participants  were  eighty  fourth  grade  students  comprising  three  self  contained  classrooms  in  an 
elementary  school  in  a  suburban  school  district.  The  study  was  conducted,  with  the  cooperation  of  the 
participating  teachers,  in  the  students'  regular  classrooms  during  the  school  day.  Parental  consent  had  been 
obtained,  and  students  were  aware  of  the  study.  All  students  were  required  to  complete  the  materials  used  in 
the  study  as  part  of  their  normal  social  studies  requirements. 
Procedures 

Prior  to  the  study,  all  participants  had  completed  the  assessment  of  video  game  experience  form. 
These  were  scored  and  a  median  split  was  used  to  assign  the  participants  to  two  groups  of  equal  size:  High 
Video  Game  Experience  and  Low  Video  Game  Experience. 

Equal  numbers  of  participants  within  the  High  and  Low  Game  Experience  groups  were  then 
randomly  assigned  to  the  linear  and  non-linear  treatments.  Participants  were  not  informed  of  their  group  or 
treatment  assignments.  They  were  told  that  they  were  trying  out  some  new  computer  software  which  other 
schools  might  someday  use.  Following  identical  procedures,  both  treatments  were  delivered  concurrently 
using  Macintosh  computers  located  in  the  classrooms.  The  use  of  the  Macintosh  and  mouse  was 
demonstrated  briefly  for  the  whole  class  during  their  social  studies  period,  then  each  participant  was  given  a 
sealed  packet  containing  a  disk  and  activity  sheet.  Participants  each  completed  the  treatment  individually, 
under  classroom  teacher  supervision,  during  the  next  three  school  days. 
While  using  the  hypertext  at  the  computer,  each  student  completed  the  activity  sheet. 

As  each  student's  thirty  minute  work  time  elapsed,  activity  sheets  and  disks  were  collected  and  the 
student  completed  the  attitude  survey  which  was  also  collected.  Several  days  after  the  lesson  was 
completed,  the  delayed  post  measure  was  administered  by  the  researcher  during  the  class  social  studies 
period.  After  all  data  had  been  gathered,  a  debriefing  session  was  held  in  each  classroom  and  results  were 
shared  with  the  students  and  teachers. 

Results 

Worksheet  Scores,  Post-Lesson  Scores,  and  Attitudes 

Each  of  the  three  dependent  measures  was  subjected  to  separate  two  way  ANOVA's  with  Video 
Game  Experience  and  Treatment  as  independent  variables.  No  significant  main  effects  or  interactions 
were  found  for  any  of  the  dependent  variables. 
Exploratory  Analysis 

A  post  hoc  analysis  of  the  data  for  each  dependent  variable  was  completed  using  a  three  way 
ANOVA  with  Gender  as  the  added  independent  variable.  A  significant  main  effect  was  found  for  Gender 
Ed  ,72)  =  4.0,  p<.05  on  the  Likert  scale  attitude  score  for  lesson  organization.  Males  (mean  =  3.42) 
rated  the  lesson  organization  as  clearer  and  more  easy  to  follow  '.han  did  females  (mean  =  2.91 ). 

Discussion 

The  study  explored  several  potential  extensions  to  the  experimental  designs  previously  used  in 
studies  comparing  linear  and  non-linear  hypertext:  (a)  younger  subjects,  (b)  grouping  by  video  game 
experience,  (c)  assigned  instructional  activity  rather  than  computerized  lesson,  (d)  worksheet  rather  than 
achievement  test,  (e)  stand-alone  classroom  computer  for  lesson  delivery  rather  than  mass  computer  lab. 
(f)  recall  post-measure  rather  than  delayed  post-test.  While  the  study  did  not  find  a  significant  effect  for 
any  one  of  these  factors,  the  experience  gained  with  these  extensions  may  assist  in  the  design  of  future 
studies  in  this  emerging  area. 

That  fourth  grade  students  i-sing  both  the  linear  and  non-linear  versions  successfully  completed 
the  activity  sheet  supports  the  asseition  that  lesson-length  hypertexts  can  be  used  effectively  by  learners 
at  this  level.  This  finding  is  supported  by  the  recently  published  Shin,  Schallert,  and  Saveyne  study 
( 1994).  Student  attitudes  toward  both  lesson  versions  remained  moderately  positive  (1-5  Likert  scale, 
mean  =  3.475)  after  completing  them.  In  addition,  the  experimenter  observed  that  most  subjects 
appeared  to  approach  the  lesson  with  enthusiasm.  These  results  appear  to  justify  further  investigation 
of  hypertext  applications  at  the  elementary  school  level. 

That  the  video  game  experience  factor  did  not  yield  significant  results  may  be  an  indication 
that  the  two  media,  video  games  and  hypertext,  are  too  dissimilar  for  skills  or  attitudes  to  transfer 
between  them.  It  is  possible  as  well  to  question  the  validity  of  the  instrument  used  in  this  study  to 
measure  video  game  experience.  Information  gained  in  the  process  of  development  showed  it  to  be 
difficult  to  assess  a  general  level  of  "video  game  experience"  beyond  expertise  with  individual  games  or 
game  systems.  Further  investigation  of  the  influence  of  experience  factors  on  hypertext  use  has  been 


suggested  by  'he  results  of  other  studies  (Lin,  1991 ;  Kahn  and  Landow,  1993).  Since  the  selection  of 
video  game  k  .inology  as  the  experience  factor  for  this  study  was  somewhat  arbitrary,  the  effects  of 
experience  with  other  video  or  microcomputer  applications  on  hypertext  use  might  be  investigated  in 
future  studies. 

The  participating  teachers  indicated  that  the  employment  of  the  hypertext  as  an  assigned 
activity  supplementary  to  classroom  teacher-directed  instruction  was  appropriate  to  this  particular 
curriculum  area  and  to  their  general  classroom  practices.  One  potentially  confounding  factor  with  this 
approach  is  the  possibility  that  variation  among  teachers  in  the  implementation  of  the  curriculum 
content  related  to  that  presented  in  the  hypertext  might  affect  students'  performance  in  the  experimental 
activity.  Future  studies  may  want  to  control  for  this  potential  influence,  perhaps  by  using  a  pre-test  to 
determine  prior  content  knowledge.  Another  possible  approach  would  be  t  use  hypertext  whose 
content  had  previously  been  introduced  to  all  subjects  via  uniform  readings  or  a  video  presentation. 
The  hypertext  activity  would  then  be  used  in  a  role  similar  to  that  suggested  by  some  researchers  for 
instructional  simulations  (Thomas  and  Hooper,  1991 ;  Thurman,  1993),  to  encourage  transfer  and 
application  rather  than  acquisition  of  new  knowledge. 

The  activity  sheet,  when  used  in  the  experimental  setting,  appeared  to  be  more  difficult  to 
complete  than  the  development  and  pilot  results  had  indicated.  It  is  possible  that  the  length  and 
difficulty  of  the  activity  (24  items,  mean  =  10.38)  may  have  caused  some  subjects  to  abandon  the  task 
prematurely.  Attempting  to  keep  the  lesson  content  appropriate  to  the  subjects'  age,  resulted  in  a 
relatively  short  (30  minute)  treatment  time.  A  treatment  of  this  duration  may  not  have  been  sufficient 
to  elicit  measurable  differences  in  treatment  effect,  an  issue  raised  recently  by  Reeves  (1993)  and 
relevant  as  well  to  some  of  the  previously  cited  instructional  hypertext  studies.  Future  studies  might 
employ  treatments  of  longer  duration. 

Delivering  the  experimental  treatment  throughout  the  school  day  via  individual  computers 
located  in  the  classroom  offered  both  advantages  and  disadvantages.  The  configuration  and  schedule 
appeared  to  be  accepted  by  students  and  teachers  as  conforming  with  routine  classroom  practice.  The 
experimental  procedures  appeared  to  run  smoothly  as  part  of  the  classroom  routine,  providing  a 
seemingly  uniform  and  non-threatening  experience  for  students  regardless  of  treatment,  classroom,  or 
group  assignment. 

On  the  other  hand,  subjects  experienced  the  experimental  treatments  over  the  course  of  three 
school  days.  Ongoing  activities  within  the  classroom  and  the  school  during  this  time  may  have 
resulted  in  unequal  experimental  conditions  for  some  subjects.  One  student,  for  example,  may  have 
worked  as  the  only  student  in  the  classroom,  while  another  may  have  worked  at  the  computer  while 
potentially  distracting  class  activities  went  on  nearby.  Choosing  between  the  authenticity  of  the 
classroom  setting  and  the  more  controlled,  but  perhaps  more  artificial,  computer  lab  setting  is  difficult. 
Perhaps  using  small  groups  of  classroom-based  computers,  thus  allowing  the  treatment  to  be  delivered 
during  one  class  period  over  several  days,  or  during  one  full  school  day,  might  provide  a  satisfactory 
compromise. 

The  relatively  low  activity  sheet  scores  and  lack  of  detailed  information  about  subjects'  prior 
knowledge,  made  interpretation  of  post-measure  scores  difficult.  In  the  future,  this  measure  could  be 
paired  with  a  more  typical  post-test,  or  compared  with  a  broader  test  of  prior  knowledge.  The  approach 
attempted  by  this  post  measure  is  congruent  with  that  of  some  hypertext  studies  published  since  its 
design  (Jonassen  and  Grabinger,  1993;  Jonassen  and  Wang,  1993).  Such  studies  might  provide 
guidance  for  designing  more  sensitive  instruments  for  measuring  levels  of  structural  knowledge 
acquisition  and  retention. 

Since  none  of  the  previous  hypertext  studies  had  found  any  achievement  or  attitude  differences 
based  on  gender,  the  gender  difference  in  attitude  toward  lesson  organization  found  in  this  study  was 
unexpected.  This  finding  is  not  surprising,  however,  considering  that  girls  have  been  shown  to  use 
computers  less  outside  of  school,  and  are  more  likely  than  boys  to  self-select  out  of  computing 
activities  in  school  (Gilliland,  1990).  The  significant  difference  in  attitude  found  here,  coupled  with  no 
difference  in  achievement  by  gender,  might  merit  further  investigation  in  light  of  issues  raised  about 
innate  vs.  socialized  gender  differences  related  to  information  technology  (Kirk,  1992). 

It  might  be  fruitful,  as  well,  to  investigate  how  interactions  between  experience  level  and 
gender  might  influence  students'  attitudes  toward  a  hypertext  lesson.  Canada  and  Brusca  (1991)  report 
that  gender-based  attitude  differences  toward  technology  disappear  when  males  and  females  have  equal 
amounts  of  technology  experience.  While  this  study  did  not  find  significant  attitude  differences  by 
experience  level,  membership  in  the  High  Video  Game  Experience  Group  was  two-thirds  male  (24  of 
36  members  were  male),  and  the  Low  Experience  Group  two-thirds  female  (25  of  44  members  were 


ERIC 


396 


female).  Future  studies  might  measure  experience  with  various  technologies  by  gender  for  comparison 
with  attitude  scores.  Another  approach  could  be  to  investigate  the  nature  of  the  attitude  differences  in 
greater  detail,  perhaps  employing  an  interview  or  other  more  sensitive  measure  of  attitude. 
Conclusions 

Although  knowledge  acquisition  or  retention  benefits  have  not  been  demonstrated  for  the 
small-scale  hypertext  employed  in  this  study,  it  is  clear  that  this  type  of  hypertext  can  be  used  by 
elementary  school  students.  This  finding  justifies  further  investigation  of  effective  instructional  uses 
for  hypertext  at  this  level.  The  attempt  in  this  study  to  create  an  authentic  classroom  setting, 
instructional  task,  and  measure  of  student  success,  may  provide  inspiration  or  guidance  for  the  design  of 
future  studies  in  this  area. 

If  there  are  learner  characteristics  which  influence  success  with  hypertext,  they  remain  to  be 
identified.  The  exploratory  analysis  done  here  suggests  that  influences  of  gender  on  attitude  and 
performance  in  this  area  might  be  investigated  further. 
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Introduction 


Teachers  underutilize  computer  technology  in  teaching.  At  the  same  time,  there  is  much  support  for 
the  use  of  computers  in  education.  Strong  evidence  for  this  is  the  increasing  number  of  computers  available 
per  student — the  microintensity  level.  The  national  average  has  improved  to  18  students  per  computer  as  of 
1992  (Market  Data  Retrieveal).  Yet,  teachers  have  not  overwhelmingly  adopted  computer  technology  for 
teaching.  According  to  a  national  surevy,  only  one  teacher  per  school,  on  average,  integrates  computer 
technology  into  his  or  her  teaching  (Sheingold  &  Hadley,  1990). 


Questions 

This  apparent  discrepancy  between  teachers'  use  of  computers  and  others'  expectations  of  teachers'  use 
of  computers  broached  several  questions  focusing  on  the  developmental  history  of  teachers  beginning  with 
their  undergraduate  training. 

1 .  At  what  levels  do  student-teachers  expect  to  use  computers  during  professional  teaching? 

2.  At  what  levels  does  the  same  population  actually  use  computers  after  having  taught 
professionally  for  one  year? 

3 .  Do  any  personal  variables  predict  levels  of  use  during  either  condition? 


Method 

A  longitudinal  study  of  teachers'  computer  use  and  personal  variables  which  might  predict  that  use 
was  undertaken.  The  study  was  begun  with  a  group  of  undergraduate  elementary  majors  during  their 
penultimate  year  of  study  and  was  repeated  three  years  later  after  the  same  participants  had  graduated  and 
worked  as  teachers  professionally  for  one  year. 

Participants  responded  in  writing  to  questionnaires  that  included  several  measures.  Similar  formats 
were  used  in  both  phases  with  the  exception  that  for  the  first  phase  the  participants  were  instructed  to 
respond  to  the  questionnaires  based  on  their  expectations  while  during  the  second  phase  they  were  instructed 
to  respond  based  on  their  actual  experience.  Also,  during  the  second  phase,  the  questionnaires  were  mailed  to 
the  participants  rather  than  the  questionnaires  being  administered  simultaneously  in  a  classroom.  Upon  the 
first  mailing,  the  number  of  respondents  was  poor.  A  follow  up  of  telephone  calls  to  all  original 
participants  greatly  increased  the  number  of  responses. 


Participants 

During  the  first  phase,  167  preservice  undergraduate  elementary  education  majors  participated 
(Marcinkiewicz  &  Grabowski,  1992).  During  the  second  phase,  100  of  the  original  participants  participated. 
Attrition  was  'ue  to  untraceable  change  of  address,  failure  to  respond,  or  change  of  profession. 


Variables 

The  criterion  variable  was  levels  of  computer  use.  The  personal  variables  were  innovativeness,  self- 
competence  in  using  computers  for  teaching,  perceived  relevance  of  computers  for  teaching,  and  teacher 
locus  of  control.  Demographic  data  for  age,  years  of  computer  experience,  and  gender  were  also  collected. 

Instruments 

The  instruments  used  included  the  following: 

1 .  Levels  of  Computer  Use  (Marcinkiewicz  &  Welliver,  1 993)  scale  based  on  the  model  of 
Instructional  Transformation  (Rieber  &  Welliver,  1989). 

2.  The  Innovativeness  Scale  (Hurt,  H.  T.,  Joseph,  K.,  &  Cook,  C.  D.,  1977). 

3.  Self-competence  in  computer  use  for  teaching  measures  were  developed  for  the  study. 

4.  Perceived  relevance  of  computer  use  to  teaching  measures  were  developed  for  the  study. 

5.  Teacher  Locus  of  Control  Scale  (Rose,  J.  S.,  &  Medway,  F.  J.,  1981,  1982). 
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Analyses  and  Results 

Results  of  the  LCU  scale  were  compared,  logistic  regression  was  calculate  to  identify  predictors  of 
levels  of  computer  use. 

a.  Phase  1  data  analysis  revealed  97%  expected  to  use  computers  in  teaching  and  only  2.7%  did  not 
expect  to  use  computers  at  all. 

b.  Phase  2  revealed  61  %  actually  used  computers  and  39%  did  not  use  computers  at  all  for  teaching, 
c     Self-competence  and  perceived  relevance  predicted  computer  use  in  the  first  phase  but  not  the 

second. 

d.  The  correlation  between  perceived  relevance  and  self-competence  nearly  doubled  between  phases. 

e.  Locus  of  control  from  the  first  phase  predicted  computer  use  in  the  second,  c2  (1,  n  =  150)  =  4.5, 
p  <  .03. 

f.  Phase  2  actual  computer  use  data  were  lower  than  Phase  1  expectations  but  were  somewhat  better 
than  that  of  an  unrelated  group  (N=  170).of  practicing  teachers  (Marcinkiewicz,  1993). 

See  Table  1 . 


Table  1: 

Computer  Use:  Phase  1  vs.  Phase  2  vs.Other  Practicing  Teachers' 

Levels  of  Computer  Use         Phase  1  Expectations  Phase  2  Actual  Other  Practicing 

 Teachers'  Actual 

Nonuse  2J  39  ~~  ~~ 43 
Utilization  84  60  49 
Integration  !  2  


Discussion 


Overall  the  participants'  actual  computer  use  was  lower  that  the  ir  undergraduate  expectations,  but 
somewhat  better  than  that  of  an  urelated  comparison  group  of  practicing  teachers.  The  differences  between 
the  expected  and  actual  use  invite  several  explanations.  The  levels  for  actual  use  could  have  been  a 
regression  to  the  mean,  on  the  other  hand,  when  the  levels  of  actual  use  were  compared  with  actual  levels  of 
the  unrelated  group  the  former  did  better.  They  a  significantly  higher  percentage  of  computer  users  and  a 
lower  percentage  of  computer  non-users.  The  high  expectations  of  computer  use  during  the  initial  phase 
could  be  attributable  to  a  the  universtiy  teacher  training  program.  While  no  personal  variables  were 
sustained  as  contributors,  the  increase  in  correlation  between  perceived  relevance  and  self-competence  in  the 
use  of  computers  for  teaching  suggests  their  collinearity.  The  identification  of  tea-her  locus  of  control  from 
phase  1  as  a  predictor  of  computer  use  during  phase  2  may  have  practical  application  in  guiding  student 
teachers  towards  integrating  educational  computing. 
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Abstract 

The  high  school  graduation  rate  will  increase  to  90%,  and  every  adult  American  will  be  literate  and 
possess  the  knowledge  and  skills  necessary  to  compete  in  a  global  economy  are  two  of  six  national 
education  goals  advocated  in  "America  2000."  At  this  time  the  majority  of  high  school  drop-outs  are 
considered  seriously  deficient  in  reading,  English,  mathematics,  and  other  functional  skills.  Consequently, 
our  nation's  competitiveness  in  the  global  economy  is  weakened  and  the  burden  of  social  welfare  programs 
has  been  increasing.  The  earlier  reading  improvements  can  be  accomplished,  the  better  the  chance  to  increase 
high  school  graduation  rates. 

This  study  addresses  regulating  closed-captioned  videotape  prompt  rates  as  a  technological  approach 
to  improve  reading  comprehension/retention  skills  in  "at  risk"  elementary  school  students.  It  is  a  within 
factor  experimental  design  which  examines  retention  of  learning  from  closed-captioned  videotape  with 
regulated  prompt  rates.  Two  groups  comprised  of  158  fourth,  fifth,  and  sixth  grade  reading  deficient  students 
(i.e.,  Chapter  1  ar.d  mildly  disabled)  participated  in  a  Pilot  Study  (18  students)  and  Experimental  Study  (140 
students).  Students  were  randomly  assigned  to  either  an  Average-Paced  Closed-Captioned  Video,  a  Slow- 
Paced  Closed-Captioned  Video,  or  printed  text  (no  video),  which  served  as  a  control  measure. 

Results  indicate  significantly  more  learning  occurs  for  those  students  using  captioned  video 
compared  to  those  having  traditional  print  materials.  Additionally,  students  assigned  to  the  Slow-Paced 
Prompt  Rate  retained  significantly  more  information  than  those  having  the  Average-Paced  captioned  video. 

These  results  suggest  educators  can  better  help  their  reading  deficient  students  by  choosing 
captioned  video  curriculum  other  than  traditional  print  materials.  Results  also  suggest  that  video  producers 
should  take  into  consideration  the  prompt  rate  of  their  captioned  video  materials  and  implement  captioned 
prompt  rates  that  are  appropriately  paced  for  use  in  inclusive  classroom  environments. 


Introduction 

Title  1/Chapter  1  of  the  Elementary  and  Secondary  Education  Act  (ESEA),  enacted  in  1965,  and 
PL  94-142,  enacted  in  1975,  have  been  the  driving  forces  providing  extra  instruction  in  compensatory 
programs  in  reading,  writing,  and  mathematics  to  millions  of  disadvantaged,  "at  risk"  children,  and  mildly 
disabled  children.  For  both  programs,  there  is  iittle  consistency  within  and  between  states  as  to  method  of 
instruction,  materials,  and  mode  of  evaluation  (LeTendre,  1991;  Johnson,  1987).  As  early  as  1979  Drum 
and  Calfee  reported  reading  compensatory  programs  resembled  "regular"  reading  instruction  and  suggested 
the  programs  only  added  amount  of  instruction  without  changing  the  manner  of  instruction.  LeTendre 
(1991)  comments  that  most  Chapter  1  reading  programs  still  rely  on  traditional  practices  (i.e.,  ditto  sheets, 
questions  at  the  end  of  chapters,  etc.). 

Despite  these  efforts,  compensatory  reading  programs  are  not  highly  successful  in  bringing 
students  to  reading  levels  of  their  more  advantaged  peers.  In  an  extensive  U.  S.  Department  of  Education 
study,  the  report  concluded  that  "students  receiving  Chapter  1  services  experience  larger  increases  in  their 
standardized  achievement  test  scores  than  comparable  students  who  do  not.  However,  their  gains  do  not 
move  them  substantially  toward  the  achievement  levels  of  more  advantaged  students"  (cited  in  Fagan  & 
Heid,  1991).  The  level  of  reading  improvement  for  mildly  disabled  students  is  equally  dismal  (Levine, 
1987).  They  never  catch-up  with  their  nondisabled  peers.  Chapter  1  and  reading  programs  provided  for 
mildly  disabled  students  are  closely  related  in  that  both  programs  are  presently  under  close  scrutiny  for 
accountability  for  student  performance.  It  is  estimated  the  majority  of  students  who  drop  out  of  school  have 
severe  reading  deficits,  despite  receiving  Chapter  1  or  services  in  reading  for  learning  disabilities  (LeTendre, 
1991;  Johnson,  1987). 

The  reauthorization  of  Chapter  1  by  the  Hawkins-Stafford  School  Improvement  Amendments  of 
1988  (U.S.  House  of  Representatives  Committee  on  Education  and  Labor)  mandates  accountability  for 
reading  improvements;  the  new  "transition"  amendment  to  Individuals  with  Disabilities  Act  (IDEA) 
essentially  raises  the  same  issues  (Sherman  &  Sherman,  1989).  Latest  figures  suggest  approximately  51% 
of  mildly  disabled  young  adults  are  without  employment  after  receiving  intensive  special  educational 
services  as  students.  The  majority  of  these  adults  are  poor  readers  (Behrmann,  1992). 

Clearly  there  is  encouragement  for  educators  to  explore  new  methods  and  materials  that  increase 
student  performance  while  stressing  high  order  thinking  (Hofmeister,  1992).  Chapter  1  and  IDEA  can  play 
an  important  role  in  the  pursuit  of  two  of  our  national  education  goals:  to  make  sure  that  by  year  2000  (1) 
the  high  school  graduation  rate  will  increase  to  90%,  and  (2)  every  adult  American  will  be  literate  and  will 
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possess  the  knowledge  and  skills  necessary  to  compete  in  a  global  economy  and  exercise  the  rights  and 
responsibilities  of  citizenship  (Bush,  1991). 

Use  of  closed-captioned  educational  video  can  be  a  promising  technological  method  to  accomplish 
Education  2000  goals.  Recent  results  of  a  study  using  closed-captioned  video  with  college  level  students 
having  reading  deficits  suggest  closed-captioned  video  may  be  an  effective  method.  Preliminary  results 
suggest  students  without  reading  deficits  learn  and  retain  information  more  efficiently  after  viewing  closed- 
captioned  educational  video  than  without  closed-captioning;  however,  this  proved  to  be  the  least  effective  for 
those  students  with  reading  deficits  (Meyer  &  Lee,  1992).  These  effects  may  be  the  result  of  the  prompt 
rate,  which  is  matched  with  the  speed  of  the  narration.  Poor  readers  may  not  be  able  to  keep  up  with  the 
rate;  therefore,  the  captioning  may  be  frustrating  and  distracting  to  the  reading-deficient  student.  Therefore, 
there  is  a  need  to  examine  the  effect  of  the  prompt  rate  of  closed-captioned  educational  video  to  better 
understand  how  it  affects  the  reading  comprehension  of  Chapter  1  and  learning  disabled  students. 

As  reported  in  the  National  Captioning  Institute  Newsletter  (August,  1990),  former  First  Lady 
Barbara  Bush,  who  made  literacy  one  of  her  personal  projects,  voiced  support  for  using  captioning  as  an 
educational  tool.  Following  a  demonstration  of  closed-captioned  technology  at  a  White  House  meeting, 
Mrs.  Bush  commented,  "It  is  so  exciting  to  learn  about  research  that  indicates  captioned  television  can  help 
adults  and  children  improve  their  reading  skills,  both  at  home  and  in  a  classroom."  Mrs.  Bush  was 
cctimenting  how  closed-captioned  video  helps  Asians  and  Hispanic-Americans  learn  English.  This  research 
explores  whether  these  same  kinds  of  gains  can  be  made  with  persons  who  have  significant  reading  deficits, 
but  have  English  as  their  native  language.  If  they  respond  to  this  technological  tool,  there  is  great  potential 
to  use  closed-captioned  video  throughout  American  school  systems,  from  elementary  school  through  adult 
education. 

Much  of  the  current  research  with  closed-captioned  video  has  been  done  with  Hearing  Impaired  (HI) 
populations  (Montandon,  1982;  Sherman  &  Sherman,  1989)  and  with  persons  who  use  English  as  a  Second 
Language  (ESL)  (Markham,  1989;  Spanos  &  Smith,  1990).  Results  are  encouraging  that  closed-captioning 
is  effective  in  enhancing  learning.  At  this  time,  however,  little  has  been  done  investigating  whether  reading 
deficient  students  being  served  in  compensatory  reading  programs  can  learn  more  efficiently  using  closed- 
captioned  video  media  with  prompt  rates  correlated  to  their  comprehension  reading  rates. 

This  study  investigates  whether  reading  deficient  Chapter  1  and  mildly  disabled  students  (i.e.,  those 
with  learning  disabilities  and/or  behavioral  disorders)  learn  and  retain  information  more  efficiently  using 
closed-captioned  video  with  appropriate  paced  prompt  rates.  Positive  results  using  captioning  technology 
should  spark  a  new  "industry"  in  the  tailoring  of  closed-captioned  videotaped  educational  media  for  learners 
of  all  abilities,  including  those  with  various  reading  deficits. 

The  first  direct  result  of  this  research  should  affect  methods  and  materials  in  the  nation's  Chapter  1 
programs  and  those  designed  for  mildly  disabled  reading  students,  including  those  served  in  inclusion 
models.  Positive  results  from  this  study  will  support  future  directions  to  ensure  accountability  for  optimal 
learning  for  Chapter  1  and  learning  disabled  reading  students.  Development  and  use  of  more  appropriate 
closed-captioned  video  across  disciplines  v.  ',1  better  ensure  success  for  all  students  and  enhance  the 
probability  of  improved  high  school  retention/graduation  rates  as  well  as  increased  literacy  skills  in 
America. 


Method 

The  purpose  of  this  study  is  to  examine  closed-captioned  prompt  rates  and  their  effect  on  learning 
for  elementary  students  who  underachieve  in  reading. 

Subjects 

Seventy-eight,  fourth,  fifth,  and  sixth  grade  students  in  Chapter  1  reading  and  sixty-two  students 
with  learning  disabilities  (52  students)  anu/or  behavioral  disorders  (10  students)  participated  in  this  study. 
All  students  met  state  requirements  for  services  in  reading.  Criteria  for  selected  students  were  program 
placement,  reading  composite  scores  from  the  Iowa  Test  of  Basic  Skills  (ITBS),  and  Reading  Rate  and 
Accuracy  Level  tests  (Carver,  1987a,  1987b,  1992a,  1992b).  The  ITBS  reading  composite  grade 
equivalency  mean  score  was  3.64  (SD  1.03).  Carver  Reading  Rate  words  per  minute  mean  score  for  selected 
students  was  1 16.49  (SD  25.36).  Accuracy  Level  mean  score  for  selected  students  was  21.68  (SD  6.02), 
with  a  mean  grade  equivalency  of  3.23  (S_D  .97). 


Table  1:  Demographic  Composition 


Group 

Age  in  Months 

Gender 

Race 

Mean        Std.  D. 

Male  Female 

White        Black  Other 

MD 

CH-1 
Total 

135.89  (16.13) 
135.32  (12.29) 
135.57  (14.07) 

42  20 
31  47 
73  67 

49            12  1 
33            45  0 
82             57  1 

Materials 

A  10  minute  edited  educational  video,  "The  Truth  About  Turtles  (Stouffer,  1990)"  was  used  as 
treatment  material.  The  script  was  modified  to  match  the  editing.  The  video's  narration  was  replaced  with 
background  music;  therefore,  students  had  to  read  the  captioned  text  with  no  auditory  narration.  The  script 
was  captioned  and  matched  to  the  visual  image.  Two  treatment  videos  were  produced:  Average-Paced 
Prompt  Rate  and  Slow-Paced  Prompt  Rate. 


Measures 

A  13-item  multiple  choice  test  was  constructed  to  measure  students'  knowledge  of  the  topic  and  the 
content  of  the  video.  The  items  were  constructed  to  test  subjects'  ability  in  decoding  single  words, 
understanding  vocabulary,  interpretation  of  sentences  (including  appreciation  of  morphology  and  syntax), 
identifying  main  ideas,  identifying  supporting  details,  rejecting  irrelevant  or  distracting  information, 
retelling  a  passage,  identifying  the  author's  intention  and/or  point  of  view,  and  summarizing.  These  are 
considered  relevant  parameters  of  reading  comprehension  necessary  for  reading  success  (Levine,  1987).  This 
paper  and  pencil  pretest  was  read  to  the  students  in  small  groups.  Any  student  who  scored  70%  or  better  was 
to  be  eliminated  from  the  study.  No  student  scored  that  high.  This  test  served  as  the  pretest,  posttest,  and 
retention  test  measure. 


Design 

The  experimental  design  is  a  2x3x3  (Group  x  Treatment  x  Test  Scores)  within  factor  design.  The 
two  groups  of  students  are  Chapter  1  reading  students  and  mildly  disabled  students  with  reading  deficits. 
The  three  measures  are  pretest,  posttest  and  retention  test.  The  three  treatments  and  their  descriptions  are: 

A         Average-Paced  C'osed-Captioned  Video. 

Students  receiving  this  treatment  viewed  a  closed-captioned  video  with  prompt  rate  set  ai  "ie  mean 

reading  rate  of  1 16  words  per  minute. 
S  Slow-Paced  Close-Captioned  Video. 

Students  receiving  this  treatment  viewed  a  closed-captioned  video  with  prompt  rate  set  at  the  mean 

reading  rate  of  78  words  per  minute. 
P         Printed  Media. 

Students  receiving  this  treatment  read  a  printed  text  in  the  amount  of  time  allowed  for  the  closed- 
captioned  video.  (This  treatment  served  as  control.) 

Based  on  the  grouping  and  treatment  assignment  described  above,  a  grouping  chart  is  depicted  as 

follows: 


Table  2:  Design  Chart  and  Cell  Size 


Caption  Prom 

pt  Rate 

Printed 

Average-Paced 

Slow-Paced 

Group 

MD 

21 

19 

22 

CH-1 

26 

20 

32 

406 


Experimental  Questions 


1 .  Between  groups,  do  Chapter  1  reading  students  perform  differently  from  mildly  disabled  reading 
students  in  terms  of  their  Carver  reading  comprehension  rate  and  accuracy  scores. 

2.  Between  groups,  do  Chapter  1  reading  students  perform  differently  from  mildly  disabled  reading 
students  in  terms  of  their  mean  pretest,  posttest  and  retention  test  scores? 

3 .  Between  groups,  do  Chapter  1  reading  students  and  mildly  disabled  reading  students  receiving  the 
same  treatment  perform  differently  in  their  mean  posttest  and  retention  test  scores? 

4.  Within  groups,  are  there  differences  among  treatments  in  mean  posttest  and  retention  test  scores? 

5 .  Are  there  differences  between  the  experimental  treatment  (captioned  video)  and  control  (print)  in 
mean  posttest  and  retention  test  scores? 

6.  Are  there  differences  between  Average-Paced  and  Slow-Paced  Prompt  Rate  treatments  in  mean 
posttest  and  retention  test  scores? 

Apparatus 

Production  equipment  used  for  this  study  were:  2  NEC  PC-VCRs,  a  Softtouch  closed-captioned 
encoding  interface  card,  a  DE 132  decoder  card,  a  Timebased  video  signal  corrector,  and  a  captioning  software 
package,  CPC  7000,  installed  on  an  IBM  compatible  personal  computer  to  serve  as  a  closed-captioned 
encoding  station.  Equipment  used  for  data  collection  were  3  AV  carts  each  housing  a  TeleCapiion  4000 
closed-captioned  decoder,  a  Sharp  25"  color  video  monitors,  and  an  RCA  4-head  video  cassette  player. 

Procedures 

Reading  rate  and  accuracy  reading  level  for  nondisabled  fourth,  fifth,  and  sixth  graders  were  147 
words  per  minute,  or  4.4  grade  level  equivalency  (Carver,  1987a,  1987b,  1992a,  1992b).  These  reading 
scores  were  established  through  standardized  tests  using  only  printed  materials.  This  study  used  materials 
demanding  attention  to  audio  and  visual  movement  as  well  as  printed  text  (i.e.,  captioning).  Other 
captioned  videos  designated  for  use  in  elementary  classrooms  were  viewed  and  measured.  Rates  ranged  from 
approximately  1 10  to  130  words  per  minute.  Therefore,  the  captioned  prompt  rate  was  set  at  1 16  words  per 
minute  as  suggested  by  the  Carver  data.  During  the  Pilot  Study,  a  questionnaire  was  given  the  students 
asking  for  their  responses  concerning  their  comfort  level  with  the  prompt  rate.  Students  indicated  it  was 
too  fast.  Therefore,  we  decided  to  use  the  prompt  rate  of  1 1 6  words  per  minute  as  the  Average-Paced 
Prompt  Rate  that  would  be  appropriate  for  nondisabled  readers.  (This  rate  also  corresponded  with  other 
commercially  produced  captioned  videos  designed  for  elementary  classrooms.) 

The  Slow-Paced  Prompt  Rate  was  determined  by  finding  the  lowest  Accuracy  Level  Score  and 
Reading  Rate  Score  from  the  subject  pool.  According  to  Carver's  Rauding  Rate  Score  table,  the  appropriate 
words  per  minute  level  would  be  approximately  2  grade  levels  below  the  1 16  words  per  minute  average. 
This  suggested  a  Slow-Paced  Prompt  Rate  of  78  words  per  minute. 

Pilot  Study:  Eighteen  students  drawn  from  the  subject  pool  (i.e.,  9  Chapter  1;  9  mildly  disabled 
reading)  were  used  in  a  Pilot  Study.  The  Pilot  Study  consisted  of  3  students  per  cell  for  both  groups. 
Results  from  this  Pilot  Study  were  used  to  adjust  the  experimental  process. 

Experimental  Study:         One-hundred  and  forty  students  who  scored  below  70%  correct  in  the  pretest  were 
randomly  assigned  to  one  of  the  three  treatments.  One  week  after  the  students  completed  the  pretest  and  met 
the  requirements,  they  were  randomly  assigned  to  one  of  the  three  treatments.  Data  of  posttest  scores  were 
collected  immediately  after  treatment;  retention  test  scores  were  collected  two  weeks  after  treatment.  Test 
items  were  read  to  the  students  for  all  measures. 

Results 

The  dependent  variables  for  the  study  were  mean  pretest,  posttest,  and  retention  test  scores.  Test 
items  of  the  pretest,  posttest  and  retention  test  were  constructed  in  a  multiple  choice  format. 

Based  on  the  experimental  questions,  all  data  were  analyzed  using  the  following  statistical  analysis 
procedures  with  a  significance  level  set  at  a  =  .05.  Trends  are  also  reported  with  a  significance  level  of  < 
.  10.  Statistical  analyses  of  all  data  in  this  investigation  were  performed  through  the  use  of  SAS  statistical 
software. 
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Analysis  of  Variance  (ANOVA)  procedures  were  conducted  for  all  measures.  When  a  significant 
difference  was  found  in  pretest  scores,  difference  scores  rather  than  mean  scores  were  used  for  posttest  and 
retention  test  analysis. 

Mean  scores  of  students'  performances  on  pretest,  posttest,  and  retention  test  are  listed  in  Table  3. 
Difference  scores  are  included  in  text. 


Table  3:  Mean  Scores  of  Three  Measures 


Group 

Pretest 

Posttest 

Retention 

Sample  (N  =  140) 

Mean 

3.69 

6.30 

5.34 

Std.Dev. 

(1.69) 

(2.40) 

(2.20) 

MD  (n  =  62) 

Mean 

4.02 

6.75 

5.79 

Std.  Dev. 

(1.84) 

(2.62) 

(2.47) 

CH-1  (n  =  78) 

Mean 

3.43 

5.95 

4.97 

Std.  Dev. 

(1.52) 

(2.16) 

(1.89) 

Average-Paced  (n=47) 

Mean 

3.89 

6.74 

5.74 

Std.  Dev. 

(1.81) 

(2.21) 

(1.89) 

Slow-Paced  (n  =  39) 

Mean 

4.37 

7.62 

6.54 

Std.  Dev. 

(1.73) 

(2.10) 

(1.82) 

Control-Print  (n=54) 

Mean 

3.04 

4.98 

4. 1 5 

Std.  Dev. 

(1.30) 

(2.11) 

(2.12) 

MD/Average  (n  =  21) 

Mean 

4.33 

7.05* 

6.05 

Std.  Dev. 

(1.83) 

(2.63) 

(1.91) 

MD/Slow  (n  =  19) 

Mean 

4.74 

8.05 

7.05 

Std.  Dev. 

(1.97) 

(2.37) 

(2.17) 

MD/Print  (n  =  22) 

Mean 

3.09 

5.36 

4.45 

Std.  Dev. 

(1.38) 

(2.22) 

(2.61) 
5.48* 

CH-1 /Average  (n=26) 

Mean 

3.54 

6.50 

Std.  Dev. 

(1.75) 

(1.84) 

(1.87) 

CH-l/Slow  (n=20) 

Mean 

4.00* 

7.20 

6.05 

Std.  Dev. 

(1.41) 

(1.77) 

(1.28) 

CH-l/Print  (n  =  32) 

Mean 

3.00 

4.72 

3.91 

Std.  Dev. 

(1.27) 

(2.04) 

(1.71) 

*  denotes  one  missing  data. 


Between  Group  Effects 

Question  #"  1 :  A  multivariate  analysis  was  performed.  No  group  differences  were  found  for  the  Carver 
reading  rate  and  accuracy  measures.  Both  Chapter  1  and  mildly  disabled  students  were  equivalent  in  their 
reading  skills. 

Question  #2:  Analysis  for  group  differences  produced  a  significant  effect  on  pretest  test  scores  E(l,138) 
=  4.25,  p  =  .0410.  The  mildly  disabled  students  (M  =  4.02,  ST2  =  1 .84)  outperformed  their  Chapter  1  peers 
(M  =  3.43,  S_D_  =  1 .52)  in  their  previously  acquired  knowledge  about  turtles. 

No  group  differences  were  found  for  posttest  and  retention  test  measures.  Chapter  1  and  mildly 
disabled  students  performed  equally  for  all  treatments. 

Question  #3:       No  group  differences  were  found  in  posttest  and  retention  test  for  all  treatments.  Both 
groups  learned  similar  amounts  of  information. 
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Within  Group  Effects 


Question  #4:       For  mildly  disabled  students,  the  pretest  measure  was  significant  E(2,  61)  =  5.16, 
H  =  .0086.  Mildly  disabled  students  assigned  to  both  Prompt  Rates  (Average-Paced  Prompt  Rate  M  =  4.33, 
§D  =  1 .83;  Slow-Paced  Prompt  Rate  M  =  4.74,  SJD  =  1 .97)  had  more  previous  knowledge  about  turtles 
than  their  peers  (M  =  3.09, S_D  =  1 .38)  in  print  media.  No  difference  was  found  for  both  posttest  and 
retention  test  measures. 

For  Chapter  1  students,  a  trend  was  found  E(2,  77)  =  2.82,  p.  =  .0658  for  pretest.  Chapter  1 
students  assigned  to  both  Prompt  Rates  (Average- Paced  Prompt  Rate  M  =  3.54,  SD.  =  1 .75;  Slow-Paced 
Prompt  Rate  M  =  4  00,  SD  =  1 .41)  tended  to  know  more  information  about  turtles  before  treatment  than 
their  same-group  peers  in  the  control  group  (M  =  3.0,  £D  =  1.27)  using  print  media.  A  significant 
difference  was  found  for  posttest  E(2,  76)  =  4.09,  p.  =  .0206.  Chapter  1  students  assigned  to  the  captioned 
video  treatments  (Average-Paced  Prompt  Rate  Difference  M  =  2.96,  £D  =  1.82;  Slow-Paced  Prompt  Rate 
Difference  M  =  3. 1 1 ,  SD  =  1 .4 1 )  scored  higher  than  their  within-group  peers  (Difference  M  =  1  -72, 
SD  =  2.34)  who  received  print  media.  A  significant  difference  was  found  for  retention  test  F(2, 75)  =  3.28, 
p  =  .0433.  Chapter  1  students  assigned  to  the  captioned  video  treatments  (Average-Paced  Prompt  Rate 
Difference  M  =  2.00,  SD  =  1  78;  Slow-Paced  Prompt  Rate  Difference  M  =  2.00,  SJD  =  1 .63)  scored  higher 
than  their  peers  (Difference  M  =  .91,  SD  =  1.99)  who  received  print  media. 


Figure  1 :  Chapter  1  Within-Group  Differences 

4.00 


Pretest  Posttest  Diff  Retention  Dlff 


■  Average-Paced  ■  Slow-Paced      Hill  Print 


Treatment  Effects 

Question  #5:       For  Experimental  students,  a  significant  difference  was  found  E0>  138)  =  14.52, 
p  =  .0002  for  pretest.  Experimental  students  assigned  to  both  Prompt  Rates  (Combined  Prompt  Rate 
M  =  4- 1 1 .  £E>  =  I  -78)  knew  more  information  about  turtles  before  treatment  than  their  peers  in  the  control 
group  (M  =  3  04,  3D  =  1.30)  using  print  media.  Treatment  differences  were  found  in  both  posttest  and 
retention  test  scores.  There  was  a  significant  difference  in  posttest  scores  EG, 137)  =  8.79,  p.  =  .0036. 
Students  assigned  to  the  captioned  video  treatments  (Difference  Prompt  Rate  M  =  3.05,  SD  =  2.05)  learned 
more  information  than  students  assigned  to  print  media.  (Difference  M.  =  1-94,  £D_  =  2.26).  There  was  a 
significant  difference  in  retention  test  scores  E(l  ,137)  =  6.54,  p  =  .01 16.  Students  assigned  to  the  captioned 
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video  treatments  (Difference  Prompt  Rate  M  =  2.00,  SD.  =  1 .84)  retained  more  information  than  students 
assigned  to  print  media.  (Difference  M.-  I.09,S_D.=  2.30).  Figure  2  illustrates  differences  between 
combined  treatments  and  control 


Figure  2:  Experimental  vs.  Control— Difference  Means 


Pretest 


Posttest  Diff 


Retention  Diff 


Video  9  Print 


Question  #6:       No  difference  in  experimental  groups  was  found  in  pretest.  Therefore,  an  ANOVA  was 
used  for  analyses  of  mean  scores. 

A  trend  was  found  in  posttest  E(l,  84)  =  3.48,  p  =  .0655.  Students  assigned  to  the  Slow-Paced 
Prompt  Rate  treatment  (M  =  7.62,  ££>  =  2.10)  tended  to  do  better  than  those  assigned  to  the  Average-Paced 
Prompt  Rate  treatment  (M  =  6.74,  S_D_  -  2.21).  There  was  a  significant  difference  in  retention  test  scores 
E(! .  84)  =  3.91 ,  p  =  .0514.  Students  assigned  to  the  Slow-Paced  Prompt  Rate  treatments  (M  =  6.54, 
SJ2  =  1 .82)  retained  more  information  than  students  assigned  to  the  Average-Paced  Prompt  Rate  treatment 
(M  =  5.74,  SD.  =  1 .89).  Figure  3  illustrates  differences  between  the  two  experimental  treatments. 
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Figure  3:  Caption  Treatment  Effects 
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Discussion 


Provision  for  making  the  closed-caption  decoding  techniques  widely  available  was  through 
legislative  mandate,  with  the  target  users  being  those  with  hearing  impairment.  On  October  16,  1990, 
former  President  Bush  signed  the  Decoder  Circuitry  Act  into  law.  This  act  became  effective  in  1 993  and 
requires  all  new  television  sets  13"  or  larger  sold  in  the  United  States  to  have  built  in  decoder  circuitry.  As 
schools  purchase  new  and  replacement  equipment,  soon  all  televisions  will  bt  capable  of  decoding  closed- 
captioned  video.  Additionally,  commercial  producers  of  video  commonly  used  in  classrooms  for  educational 


purposes  are  in  the  process  of  captioning  all  their  new  offerings  and  file  videotapes.  Many  offer  to 
exchange  any  videotapes  owned  by  schools  without  captioning  for  ones  with  captioning. 

Inclusive  educational  environments  are  the  norm  in  most  states.  This  model  includes  children  with 
disabilities  in  the  regular  classroom  with  their  peers.  Special  services  are  delivered  by  the  teacher  and 
support  personnel  in  the  regular  classroom.  Therefore,  there  will  be  children  with  a  wide  range  of  abilities 
in  each  classroom.  Curriculum  materials  must  be  flexible  in  order  to  maximize  learning  for  these 
differences. 

Results  from  this  study  support  that  reading  problems  of  children  with  mild  disabilities,  the 
majority  being  learning  disabled,  and  those  in  Chapter  1,  are  similar.  Their  reading  rates  and  accuracy 
reading  levels  are  equivalent.  However,  children  who  are  classified  as  mildly  disabled  (i.e.,  learning  disabled 
and/or  behavior  disordered)  under  the  rules  and  regulations  of  IDEA  receive  much  more  intensive  remediation 
and  one-to-one  attention  for  their  deficits  than  children  in  Chapter  1  classes.  This  could  account  for  students 
with  mild  disabilities  knowing  more  information  about  turtles  than  their  Chapter  1  peers  before  treatment 
Mildlv  disabled  children  are  provided  with  more  educational  experiences  delivered  through  hands-on  materials 
and  manipulatives.  After  both  groups  received  the  information  about  turtles,  the  differences  disappeared. 
Chapter  1  students  learned  and  retained  more  information  about  turtles  than  their  mildly  disabled  peers. 

They  caught :up^  ^  improved  their  know^ge  base  about  turtles  significantly  more  than  mildly 

disabled  students  after  seeing  the  captioned  video  treatments.  They  might  have  found  the  captioned  video 
more  novel  and  therefore,  paid  more  attention  to  the  video  content.  The  more  traditional  print  materials 
were  not  as  stimulating  and  produced  significantly  less  learning  and  retention  of  information.  It  is 
interesting  to  note  that  the  mildly  disabled  students  did  not  show  a  significant  difference  in  their  learning 
when  using  print  materials.  The  emphasis  on  learning  strategies  in  their  educational  programs  may  have 
caused  this,  although  their  scores  for  captioned  treatments  were  higher,  but  not  significantly  so. 

There  is  controversy  among  teachers  whether  children  are  saturated  with  videos  in  the  classroom  to 
the  point  they  no  longer  pay  attention  to  content.  Results  support  this  is  not  the  case.  When  students 
experienced  a  moving  visual,  music,  and  print  at  the  bottom  of  the  screen,  they  attended  to  the  .nformation 
presented.  All  children  with  reading  deficits  learned  and  retained  more  though  captioned  video,  even  when 
there  was  no  narration  to  give  them  additional  clues  to  its  content. 

The  critical  question  in  the  study  was  the  examination  of  the  power  of  the  prompt  rate  Did  its 
speed  make  a  difference  for  children  who  struggle  in  reading?  A  strong  trend  indicated  that  the  Slow-Paced 
CpXte  was  better  for  these  children  when  assessing  learning  immediately  after  treatment  but  would 
this  learned  information  still  be  there  two  weeks  later?  The  retention  test  is  the  more  powerful  measure  of 
eLing  and  what  teachers  strive  for  with  their  students.  Yes.  Slow-Paced  Prompt  Rate  was  significantly 
Sthar! Average-Pace  Prompt  Rate.  Students  with  reading  deficits  responded  positively  when 
they  had  more  time  to  process  the  captioning  vocabulary.  ;,w«,u.= 
Results  of  this  study  are  important  for  teachers,  parents,  and  commerc.al  producers  of  v.deotapes. 
Children  will  learn  and  retain  more  from  captioned  educational  videotapes  rather  man  traditional  print 
materials  containing  the  same  information  (e.g.,  books,  workbooks  ditto  pages  etc.).  Additionally, 
educators  should  address  reading  deficits  in  Chapter  1  and  mildly  tabled  special  education  ^nte 
similarly  Chapter  1  students  appear  to  respond  strongly  to  novel  curriculum  approaches  wh.ch  supports 
Drum  and  Calfee's  (1979)  assertion  these  students  have  been  given  much  more  of  the  same  old  stuff 
When  teachers  have  the  option  of  using  a  television  with  a  captiomng  decoder  ch.p  along  with  a  closed- 
captioned  video,  they  should  use  it  in  their  classrooms  for  all  subjects.  „t;„n;no 
Parents  search  for  ways  to  help  their  children  improve  their  read.ng  skills.  Using  the  capt  oning 
option  on  their  home  television  for  closed-captioned  programs  is  an  easy,  inexpensive  way  to  help  tiie.r 
children  S  month  more  programs  are  closed-captioned  and  are  designated  by  a  closed-captioned  symbol 
in  television  guides.  A  growing  body  of  research  suggests  captioning  intended  for  hearing  >mpa,red 
populations  ,s  highly  effective  for  learning  language  and  improving  literacy  skills. 
P  P       CompanL  which  are  in  the  process  of  captioning  their  educational  vide otapes  sh ould  take  in o 
consideration  the  many  viable  uses  of  captioning  in  education.  Prompt  rates  should  be  designed  so  that 
Sen  with  various  reading  speeds  and  comprehension  skills  have  enough  time  to  read  and  process  the 
nation.  Until  technology  is  developed  that  allows  users  to  adjust  prompt  rates  to  the™  reading 
rate  level,  captioned  video  materials  must  take  into  consideration  a  variety  of  reading  competencies. 
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Introduction 

Off-campus  degree  programs  are  becoming  more  common  as  universities  recognize  the  need  to 
extend  educational  opportunities  beyond  the  campus  to  adults  who  are  unable  to  pursue  degrees  through 
traditional  means.  Off-campus  degree  programs  fit  well  within  the  mission  of  land-grant  university  colleges 
of  agriculture.  The  College  of  Agriculture  at  Iowa  State  University  recognized  the  importance  of  extending 
degree  programs  to  distant  learners,  and  began  offering  an  off-campus  master  of  agriculture  degree  in  1979. 
The  off-campus  program  expanded  to  include  a  bachelor  of  science  degree  in  1991.  The  purpose  of  the  off- 
campus  agriculture  degree  programs  is  to  provide  post-secondary  agricultural  education  opportunities  to 
persons  who  are  unable  to  or  prefer  not  to  study  on  campus  (Miller  &  Honeyman,  1993). 

Off-campus  students  are  significantly  different  from  traditional  college  students.  Distant  learners  are 
typically  older  and  generally  maintain  a  professional  career  in  addition  to  taking  courses  (Wilson,  1991). 
Miller  and  Honeyman  (1993)  described  off-campus  learners  enrolled  in  selected  agricultural  videotaped 
courses  as  being  older,  generally  farmers  or  agricultural  professionals,  and  motivated  to  enroll  in  the 
program  to  pursue  a  degree.  Lehtola  and  Boyd  (1992)  described  agricultural  distant  learners  as  self-motivated 
and  self-disciplined  while  Gulliver  and  Wright  (1989)  noted  that  distant  learners  did  not  place  a  high  degree 
of  value  on  interacting  with  other  students.  The  unique  characteristics  of  agricultural  distant  learners  may 
interact  with  program  variables  to  create  unintended  obstacles  to  off-campus  study.  Therefore,  faculty,  staff, 
and  administrators  involved  with  off-campus  programs  must  understand  and  make  accommodations  for  the 
unique  characteristics  and  preferences  of  agricultural  distant  learners. 

Students  who  pursue  degrees  through  off-campus  programs  face  a  number  of  obstacles  not 
normally  encountered  by  traditional  college  students.  Off-campus  students  often  live  too  far  from  campus 
to  attend  on-campus  classes,  generally  have  a  number  of  competing  demands  placed  on  their  time,  and  are 
concerned  with  the  costs  associated  with  college  (Hezel  &  Dirr,  1990;  Thompson,  Simonson  &  Hargrave, 
1991).  Some  authors  (Miller  &  Honeyman,  1993;  Owen  &  Hotchkis,  1991)  contend  that  videotaped 
instruction  can  be  effectively  used  to  reduce  the  negative  effects  of  obstacles  related  to  time,  costs,  and 
convenience.  Many  institutions  of  higher  education,  including  Iowa  State  University's  College  of 
Agriculture,  are  using  television  and  videocassettes  to  a  greater  extent  than  other  media  to  capitalize  on 
these  benefits  (Gunawardena,  1988;  Miller  &  Honeyman,  1993). 

Research  is  needed  to  understand  the  nature  of  graduates'  experiences  with  off-campus  degree 
programs.  Also,  research  is  needed  to  gain  further  understanding  of  the  obstacles  that  most  significantly 
impede  the  agricultural  distant  learner's  ability  to  participate  in  and  benefit  from  off-campus  study.  This 
type  of  research  will  aid  decision  makers  in  constructing  quality  off-campus  programs  that  accommodate  the 
needs  and  preferences  of  agricultural  distant  learners. 

A  number  of  studies  (Brown,  1983;  Miller  &  Honeyman,  1993;  Weeks,  1987;  Wilson,  1990) 
involving  adult  learners  indicate  generally  positive  attitudes  toward  videotaped  instruction.  What  attitudes 
were  held  by  graduates  of  the  off-campus  agricultural  degree  program,  and  what  characteristics  of  the  delivery 
media  are  most  important  for  understanding  these  attitudes? 

Development  of  the  method  and  process  for  technology-mediated  instruction  is  needed  to  advance 
distance  education  in  agriculture.  Miller  and  Honeyman  (1994)  identified  a  number  of  effective  instructional 
practices  for  videotape  that  were  espoused  in  the  literature.  What  videotape  teaching  practices  were 
important  to  graduates  of  an  agricultural  distance  education  program,  and  to  what  extent  were  the  practices 
utilized  by  college  teachers  of  agriculture? 
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Purpose  and  Objectives 


The  purpose  of  this  study  was  to  describe  experiences  of  graduates  with  an  off-campus  agriculture 
degree  program.  Additionally,  the  researchers  sought  to  describe  selected  teaching  variables  and  attitudes 
related  to  agricultural  courses  delivered  by  videotape.  The  objectives  of  the  study  were  as  follows: 

1 .  Describe  selected  variables  related  to  graduates'  experiences  with  an  off-campus  professional 
agriculture  degree  program. 

2.  Describe  the  perceptions  held  by  graduates  of  an  off-campus  professional  agriculture  degree 
program  related  to  obstacles  encountered  in  off-campus  study. 

3.  Describe  graduates'  perceptions  regarding  the  importance  of  and  the  extent  to  which  effective 
videotape  instructional  practices  were  utilized  by  instructors. 

4.  Describe  attitudes  of  graduates  toward  the  use  of  videotape  as  a  tool  for  delivering  agricultural 
courses. 

5 .  Describe  relationships  between  graduates'  attitudes  toward  videotaped  instruction  and  selected 
variables. 

Procedures 

Population 

The  population  for  the  study  consisted  of  all  persons  who  had  earned  a  bachelor's  or  master's  of 
professional  agriculture  degree  from  Iowa  State  University.  Forty-six  master's  degrees  and  seven  bachelor's 
degrees  had  been  awarded  through  fall  semester,  1993.  All  graduates  (N=53)  were  included  in  the  study. 

Instrumentation 

The  questionnaire  included  four  Likert-type  scales  in  addition  to  selected  demographic  questions  and 
questions  related  to  graduates'  experiences  with  the  off-campus  program.  Content  and  face  validity  for  the 
questionnaire  were  established  by  a  panel  of  faculty  and  graduate  students  in  agricultural  education. 

A  six  point  Likert-type  scale  with  response  categories  ranging  from  insignificant  (1)  to  significant 
(6)  was  used  to  measure  graduates'  perceptions  related  to  obstacles  faced  by  off-c?  '.ipus  students.  An  item 
pool  for  the  perception  scale  was  generated  by  interviewing  administrators,  advisors,  professors,  and 
students  associated  with  the  off-campus  professional  agriculture  degree  program.  Ten  students  enrolled  in 
the  off-campus  program  participated  in  a  field  test  of  the  instrument.  Ultimately,  13  obstacles  were  selected 
by  the  researcher  to  be  included  in  the  scale.  Cronbach's  alpha  was  calculated  to  estimate  the  internal 
consistency  of  the  scale  and  resulted  in  a  coefficient  of  .71. 

Fifteen  statements  representing  effective  videotape  instructional  practices  were  selected  from  an 
instrument  developed  by  Miller  and  Honeyman  (1994).  Each  of  the  15  statements  received  mean  ratings  of 
4.25  or  higher  for  importance  on  a  five-point  Likert-type  scale  in  the  Miller  and  Honeyman  (1994)  study. 
Graduates  were  asked  to  indicate  their  level  of  agreement  with  the  statements  regarding  the  extent  to  which 
the  practices  were  important  and  the  extent  to  which  they  had  occurred.  Data  from  the  Miller  and 
Honeyman  (1994)  study  were  used  to  estimate  the  internal  consistency  of  the  scales.  Cronbach's  alpha 
coefficients  were  .91  for  the  importance  scale  and  .87  for  the  occurrence  scale. 

The  scale  for  assessing  attitudes  toward  videotaped  instruction  consisted  of  1 1  Likert-type  items, 
with  five  response  categories  ranging  from  strongly  disagree  (1)  to  strongly  agree  (5).  The  attitude  scale 
was  previously  developed  by  Miller  and  Honeyman  (1993)  and  had  a  Cronbach's  alpha  coefficient  of  .86. 

Data  Collection 

Data  for  the  study  were  collected  by  mailed  questionnaire.  The  questionnaire,  a  cover  letter,  and  a 
stamped  return  envelope  were  sent  to  all  graduates  of  the  professional  agriculture  degree  program. 
Approximately  four  weeks  after  the  initial  package  was  mailed,  a  second  complete  package  was  sent  to  all 
nonrespondents.  Two  weeks  after  the  second  complete  package  was  mailed,  telephone  calls  were  made  to  all 
nonrespondents  to  encourage  their  participation  in  the  study.  Forty-two  master's  graduates  and  four 
bachelor's  graduates  completed  and  returned  the  questionnaire  for  a  response  rate  of  87%. 
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Analysis  of  Data 


All  data  were  analyzed  with  the  SPSS/PC+  personal  computer  program.  Appropriate  statistics  for 
description  were  used,  including  frequencies,  percents,  means,  standard  deviations,  point  biserial  correlation 
and  pearson  correlations.  All  correlation  coefficients  were  interpreted  using  Davis'  (1971)  descriptors. 

Result? 

.  Graduates  of  the  off-campus  professional  igriculture  program  ranged  in  age  from  27  to  67  years. 
The  mean  age  of  graduates  was  45  years  with  a  standard  deviation  of  9.  Approximately  89%  (41)  of  the 
graduates  were  male. 

Graduates  were  asked  to  rank  four  motivating  factors  for  enrolling  in  the  off-campus  program.  The 
graduates  rated  pursuing  a  degree  as  the  most  motivating  factor  followed  by  acquiring  current  technical 
knowledge,  enjoyment  of  learning  new  information,  and  career  advancement  (Table  1). 

The  amount  of  time  needed  to  complete  the  off-campus  degree  program  ranged  from  a  low  of  24 
months  to  a  high  of  126  months.  A  majority  (58.7%)  of  graduates  took  more  than  60  months  to  complete 
the  program  (Table  2). 

Graduates  of  the  off-campus  program  experienced  a  variety  of  delivery  mechanisms  for  their 
courses.  Several  courses  were  taught  at  different  sites  in  the  state  through  conventional  methods.  Also, 
many  students  took  courses  by  videotape  whereas  some  took  courses  via  satellite  broadcast  and  through 
two-way  audio  and  video  communications  technology.  Generally,  students  were  required  to  attend  one  or 
more  on-campus  sessions  for  each  course  even  if  the  course  was  offered  through  distance  education 
technologies.  Table  3  shows  that  a  majority  (65.2%)  of  graduates  traveled  to  campus  20  or  fewer  times. 
However,  13%  of  the  graduates  reported  traveling  to  campus  more  than  31  times. 


Table  1 

Mean  Rankings  and  Standard  Deviations  for  Factors  that  Motivated  Students  to  Enroll  in  the  Off-Campus 
Program 


Motive 

Mean 

Std.  Dev. 

Pursuing  a  degree 

1.59 

.90 

Acquiring  current  technical  knowledge 

2.52 

.99 

For  the  enjoyment  of  learning  new  information 

2.93 

1.09 

Career  advancement 

3.12 

1.17 

Table  2 

Time  in  Months  Taken  hy  Students  to  Complete  the  Off-Campus  Program 


Number  of  Times 

f 

% 

Cum  % 

13-24 

1 

2.2 

2.2 

25-36 

3 

6.5 

8.7 

37-48 

5 

10.9 

19.6 

49-60 

10 

21.7 

41.3 

61-72 

9 

19.5 

60.9 

73-84 

9 

19.5 

80.4 

85-96 

5 

10.9 

91.3 

97-108 

2 

4.4 

95.7 

109-120 

1 

2.2 

97.8 

121-132 

1 

2.2 

100.0 

Total 

46 

100.0 

100.0 

Mean  69.72        Std.  Dev.  22.77 
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On  a  six  point  Likert-type  scale,  graduates  were  asked  to  rate  the  significance  of  13  obstacles  to  off- 
campus  study.  Table  4  shows  that  10.9%  (five)  of  graduates  provided  a  mean  score  between  1.51  and  2.50 
(moderately  insignificant),  and  43.4%  (20)  provided  a  mean  score  between  2.51  and  3.50  (slightly 
insignificant).  The  remaining  45.7%  (21)  of  graduates  provided  a  mean  score  between  3.51  and  5.50 
(slightly  significant  to  moderately  significant).  The  overall  mean  score  for  the  perceived  significance  of 
obstacles  to  off-campus  study  was  3.34  (slightly  insignificant)  with  a  standard  deviation  of  .67. 


Table  3 

Number  of  Times  Respondents  Traveled  to  Campus  for  Reasons  Related  to  the  Off-Campus  Program 


Number  of  Times 

f 

%            Cum  % 

0  to  10 

18 

39.1 

39  1 

1 1  to  20 

12 

26.1 

UJ.  L, 

21  to  30 

10 

21.7 

87.0 

31  to  40 

4 

8.7 

7J.  / 

41  to  50 

1 

2.2 

97.8 

51  to  60 

1 

2.2 

I  uu.u 

Total 

46 

100.0 

100.0 

Table  4 

Overall  Mean  Scores  for  the  Perceived  Significance  of  13  Obstacles  to  Off-Campus  Studv 

Mean  f 

% 

Cum  % 

1.51-2.50 

5 

10.9 

10.9 

2.51-3.50 

20 

43.4 

54.3 

3.51-4.50 

19 

41.4 

95.7 

4.51-5.50 

2 

4.3 

100.0 

Total  46 

100.0 

100.0 

Mean  3.34     Std.  Dev.  .67 

Note:Based  on  Scale:  1 

=  insignificant;  2  =  moderately  insignificant;  3  = 

slightly  insignificant; 

4 

-  slightly  significant;  5  =  moderately  significant; 

6  =  significant. 

The  percentages  of  respondents  who  rated  each  obstacle  as  slightly  significant  to  significant  are 
presented  in  Table  5.  Four  obstacles  were  perceived  to  be  slightly  significant  to  significant  by  a  majority 
of  graduates.  The  four  obstacles  were  related  to  limited  course  offerings,  difficulty  in  balancing 
responsibilities,  access  to  library  facilities,  and  program  costs.  Obstacles  related  to  course  prerequisites, 
financial  aid,  faculty  understanding  of  student  needs,  and  dealing  with  a  number  of  departments  were  least 
significant. 
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Table  5 

Percentage  of  Respondents  Who  Selected  Slightly  Significant.  Moderately  Significant,  or  Significant  for 
Each  Obstacle 


Obstacle  % 


1 

1 . 

Limited  number  of  courses  offered. 

BZ.O 

2. 

Difficulty  in  balancing  school,  personal,  and  work 

71.7 

responsibilities. 

3. 

Lack  of  access  to  library  facilities. 

65.2 

4. 

Cost  of  the  program. 

60.9 

5. 

Attending  sessions  held  on  campus. 

47.8 

6. 

Course  offerings  did  not  fit  needs. 

47.8 

7. 

Lack  of  scholarships 

47.8 

8. 

Lack  of  access  to  instructors. 

47.8 

9. 

Lack  of  access  to  other  students. 

43.5 

10. 

Dealing  with  a  number  of  different  departments. 

39.1 

11. 

Faculty  did  not  understand  student  needs. 

37.0 

12. 

Accessing  financial  aid  at  the  University. 

34.8 

13. 

Prerequisites  required  for  classes. 

19.6 

Sixty-three  percent  (29)  of  the  42  respondents  had  taken  at  least  one  videotaped  course  through  the  off- 
campus  professional  agriculture  degree  program.  This  group  of  29  graduates  provided  data  for  the  objectives 
related  to  videotape  course  delivery.  Graduates,  on  average,  had  taken  4  videotaped  courses  with  a  range  of  1 
to  1 5  and  a  standard  deviation  of  3.86. 

Table  6  presents  means  and  standard  deviations  for  individual  items  on  the  importance  and  occurrence 
scale.  Arguably,  each  of  the  15  practices  applies  not  only  to  the  distant  setting  but  also  to  conventional 
settings.  Farr  and  Shaeffer  (1 993)  suggested  that  the  principles  of  distance  education  and  conventional 
education  are  analogous.  However,  the  15  practices  listed  in  Table  6  are  particularly  important  for  distant 
learners  (Cyrs  &  Smith,  1990;  Gibson,  1985;  Miller  &  Honeyman,  1994;  Thompson,  Simonson  & 
Hargrave,  1991;  Wilson,  1991).  Graduates  agreed  that  each  practice  was  important  for  their  learning  and 
that  each  had  occurred  in  their  videotaped  course(s),  but  graduates  provided  higher  mean  scores  for 
importance  than  for  occurrence  on  each  of  the  15  practices. 

Graduates'  attitudes  toward  videotaped  instruction  were  measured  with  a  five-point  Likert-type  scale. 
Table  7  shows  that  75.9%  (22)  of  the  graduates  provided  mean  attitude  scores  greater  than  3.5  (agree  to 
strongly  agree).  Only  3.4%  (1)  of  the  graduates  provided  a  mean  score  less  than  2.50  (disagree),  and  the 
remaining  20.7%  (6)  of  the  graduates  provided  mean  attitude  scores  between  2.51  and  3.50  (undecided). 

The  researchers  analyzed  responses  to  individual  items  on  the  attitude  scale  and  discovered  that  graduates 
provided  more  positive  responses  for  items  related  to  convenience,  opportunity  for  learning,  control  of  pace, 
and  whether  or  not  they  would  be  willing  to  take  additional  videotaped  courses.  Items  that  yielded  lower 
mean  scores  indicated  that  graduates  felt  isolated  and  would  prefer  traditional  methods  of  instructional 
delivery  over  videotape. 
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Table  6 

Means  and  Standard  Deviations  for  Videotape  Instructional  Practices  on  the  Importance  and  Occurrence  Scale 


Importance  Occurrence 

Item 


iVlCal  I 

S.D. 

1 

T^lii^  i  n  ctn  ^f>t^\r( c\  Hi*»fnr^ncl"n(       cnm  nnnn 
1  11C  1  IloLl  ULIUI  \  b)  Utll  11 alCU  lllltlllU 

4.52 

.51 

4.28 

.59 

oi  trie  rnuienui  inai  iney  were  icticning. 

oupponing  niaieriuis  ^iexi,  siuuy  guide,  cic^ 
coniriDUieu  10  my  uiiuciMtiiiuirig  01  ine  ■ 
courses(s). 

4  46 

50 

3.72 

.88 

n 

j . 

1  ne  instructor's,/  were  cninusuisiic. 

4  38 

t.  JO 

5fS 

3.90 

1.01 

4. 

The  instructor(s)  spoke  clearly. 

4.38 

.56 

4.14 

.58 

c 

J. 

Lessons  were  interesting. 

A  IS 

61 

3.62 

1.05 

6. 

The  instructor(s)  made  clear  the  relative 

4.31 

.76 

3.66 

.90 

importance  ol  the  information  presented. 

3.90 

.86 

7. 

The  instruction  required  me  to  think. 

4.31 

.47 

(5. 

1  ne  instructor's;  proviueo  a  siruciurea 

A  98 

51 

■  J  J 

3.93 

.37 

outline  oi  content  iu  ue  ictugin. 

o 

i esis  rcuuireo  tippiicuiiun  oi  ine  ^uuibc  cumcm. 

4.28 

.59 

4.07 

.75 

10. 

The  instructor(s)  spoke  at  an  appropriate  pace. 

4.24 

.58 

4.03 

.57 

n. 

The  organization  of  content  within  a 

4.24 

.58 

3.90 

.49 

given  lesson  was  logical. 

12. 

The  instructor(s)  effectively  communicated 
the  material  to  be  learned. 

4.24 

.69 

3.75 

.79 

13. 

The  instructor(s)  demonstrated  the 
interrelatedness  of  the  course  concepts. 

4.17 

.76 

3.62 

.68 

14. 

The  instructors  explained  what  I  should 
know  or  be  able  to  do  as  a  result  of  viewing 

4.14 

.64 

3.48 

.91 

the  videotapes. 

.67 

3.63 

.97 

15. 

"Real  world"  application  of  content  was 

4.1 1 

stressed  by  the  instructor(s). 


Table  7 

Overall  Mean  Scores  for  Attitude  Toward  Videotaped  Instruction 


Mean 

f 

% 

Cum  % 

1.51-2.00 

1 

3.4 

3.4 

2.01-2.50 

0 

0.0 

3.4 

2.51-3.00 

2 

6.9 

10.3 

3.01-3.50 

4 

13.8 

24.1 

3.51-4.00 

14 

48.3 

72.4 

4.01-4.50 

6 

20.7 

93.1 

4.51-5.00 

2 

6.9 

100.0 

Total 

29 

100.0 

100.0 

Mean  3.72    Std.  Dev.  .59 

Note:     Based  on  Scale:  1  =  strongly  disagree;  2=  disagree;  3=  undecided;  4=  agree;  5=strongly  agree 
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Pearson  correlations  and  point  biserial  correlations  were  used  to  describe  relationships  between 
graduate's  attitude  toward  videotaped  instruction  and  selected  variables  (Table  8).  A  low  positive 
relationship  was  found  between  attitude  and  gender.  A  low  negative  relationship  was  found  between  attitude 
and  age.  A  moderate  positive  relationship  existed  between  attitude  and  number  of  videotaped  courses  taken, 
and  a  substantial  positive  relationship  was  found  between  attitude  and  perceived  occurrence  of  the  videotape 
instructional  practices.  Students  with  more  positive  attitudes  toward  videotaped  instruction  tended  to  be 
female,  were  younger,  had  taken  more  videotaped  courses,  and  perceived  to  a  greater  extent  the  occurrence  of 
the  videotaped  instructional  practices. 


Table  8 

Summary  of  Relationships  Between  AttitudeTowMd  Videotaped  Instruction  and  Selected  Variables 


Variable 

Association 

Occurrence  of  effective  practices 

.50 

No.  videotaped  courses  completed 

.37 

Age 

-.22 

Gender3 

.22 

a  rpb 

Conclusions  and/or  Recommendations 

Most  graduates  of  the  professional  agriculture  program  took  five  or  more  years  to  complete  the 
requirements  for  their  degree  and  considered  the  limited  number  of  course  offerings  to  be  the  most 
significant  obstacle  to  off-campus  study.  More  agricultural  courses  should  be  offered  through  fiber  optics, 
satellite  broadcasts,  and  videotape  to  accommodate  the  educational  needs  of  distance  learners  and  to  facilitate 
a  more  timely  completion  of  their  degree  requirements.  Administrators  should  make  efforts  to  encourage 
greater  participation  of  college  faculty  in  the  delivery  of  off-campus  courses.  Administrators  must  recognize 
the  additional  efforts  required  of  instructors  who  teach  at  a  distance  and  consider  this  when  evaluating  the 
work  of  faculty  members. 

One  of  the  often  cited  benefits  of  distance  education  is  a  reduction  in  the  need  to  travel.  This  reduction 
of  travel  allows  students  to  better  use  their  time  in  balancing  school,  work,  and  personal  responsibilities. 
Hezel  and  Dirr  (1990)  recognized  the  importance  of  time  to  the  distant  learner  when  they  wrote,  "although 
the  term  'distance  education'  is  becoming  the  accepted  term  for  describing  new  educational  opportunities  that 
offer  students  flexibility  for  pursuing  their  degrees,  time  rather  than  distance  seems  to  be  the  major 
constraint  facing  those  students."  (p.  6)  Most  graduates  traveled  to  campus  20  or  fewer  times  for  reasons 
related  to  the  off-campus  professional  agriculture  degree  program.  Twenty  trips  to  campus  may  be 
reasonable  considering  that  most  graduates  took  five  or  more  years  to  complete  their  degree  requirements, 
but  advanced  communications  technology  must  be  exploited  further  to  reduce  the  amount  of  travel  required 
of  students.  Also,  partnerships  between  the  university  and  secondary  agriculture  teachers,  community 
colleges,  and  outlying  research  centers  should  be  established.  Students  should  be  able  to  complete 
laboratory  activities  and  obtain  academic  assistance  from  individuals  and  institutions  in  their  local 
communities. 

Overall,  the  obstacles  to  off-campus  study  were  perceived  to  be  slightly  insignificant  by  graduates  of 
the  program.  However,  a  majority  of  graduates  perceived  that  limited  course  offerings,  difficulty  in 
balancing  responsibilities,  and  program  costs  were  slightly  significant  to  significant  barriers. 
Administrators  should  attempt  to  eliminate  or  minimize  the  effects  of  each  of  these  barriers.  Data  from  this 
investigation  have  been  requested  by  and  delivered  to  persons  within  the  university  responsible  for  policy 
decisions  related  to  each  of  the  obstacles. 

Overall,  graduates  perceived  the  15  videotape  instructional  practices  to  be  important  for  their  learning 
and  perceived  that  the  practices  had  occurred  in  the  videotaped  course(s)  they  had  taken.  However,  the  extent 
to  which  the  practices  had  occurred  was  lower  than  the  perceived  importance  for  each  of  the  practices. 
College  of  agriculture  faculty  were  doing  a  relatively  good  job  of  teaching  through  videotape,  but  efforts 
should  be  made  to  help  faculty  integrate  more  of  the  effective  practices  into  their  instruction.  College  of 
agriculture  faculty  should  be  provided  an  opportunity  to  participate  in  a  formal  program  aimed  at  assisting 
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them  in  improving  the  design,  organization,  and  delivery  of  agricultural  courses  offered  via  one-way 
instructional  television.  This  assistance  should  be  based  on  sound  scholarship.  Therefore,  further  research 
is  needed  to  determine  which  teaching  methods  and  strategies  are  most  appropriate  for  specific  delivery 
systems  and  groups  of  learners.  A  study  by  Miller  and  Lehtola  (1994)  provides  an  example  of  the  type  of 
research  needed  to  further  develop  the  method  and  process  of  technology  mediated  instruction. 

A  majority  of  graduates  held  positive  to  strongly  positive  attitudes  toward  videotaped  instruction. 
Graduates  were  most  positive  about  the  opportunity  for  learning  provided  by  videotape,  the  ability  to 
control  the  pace  of  their  learning,  and  convenience.  This  conclusion  is  consisten'  with  the  work  of  Miller 
and  Honeyinan  (1993)  and  with  the  desirability  indicators  of  distance  education  that  were  identified  by 
Gulliver  and  Wright  (1989).  Videotaped  instruction  provides  a  suitable  means  of  offering  credit  courses  to 
distant  learners.  This  study  shows  that  graduates  were  more  positive  about  videotaped  instruction  when 
they  perceived  to  a  greater  extent  the  occurrence  of  effective  practices.  Therefore,  quality  instruction  is  key 
to  maintaining  the  acceptability  of  this  medium  with  distant  learners. 

The  results  of  this  study  have  direct  implications  for  college  teaching  --  particularly  distance  teaching. 
Data  from  this  study  illustrate  learning  strategies  and  preferences  of  a  select  group  of  agricultural  distance 
learners,  faculty  should  routinely  consider  the  learning  preferences  of  their  students  when  planning, 
organizing,  and  delivering  courses.  Faculty  should  also  consider  that  any  instructional  media,  including 
videotape,  will  be  more  or  less  obliging  to  a  particular  student's  learning  style  (Ullmer,  1994).  A.  challenge 
to  college  teachers  of  agriculture  is  to  discover  innovative  teaching  strategies  that  exploit  the  potential  of 
the  teaching  medium.  A  body  of  research  exists  which  demonstrates  a  significant  improvement  in  student 
achievement  and  attitudes  when  teaching  styles  are  congruent  with  learner  preferences  (Griggs,  1991). 

The  instructional  practices  listed  in  Table  6  were  considered  to  be  important  for  learning  by  the 
graduates  of  the  professional  agriculture  degree  program,  and  the  extent  to  which  graduates  perceived  the 
practices  had  occurred  was  positively  related  to  their  attitude  toward  videotaped  instruction.  Agriculture 
faculty  who  teach  at  a  distance  might  consider  doing  a  self-assessment  and  ask  their  soidents  to  assess  the 
extent  to  which  they  use  the  practices.  A  small  number  of  practices  could  be  targeted  and  consciously 
integrated  into  distance  teaching.  Feedback  on  the  extent  to  which  the  practice  was  effectively  used  could  be 
sought  from  colleagues  and  from  students  enrolled  in  their  course(s). 
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Introduction 


It  is  traditional  to  think  of  adult  learners  when  distance  education  is  mentioned.  Indeed,  the 
greatest  percentage  of  distance  students  have  been  adults.  That  tradition  is  changing. 
With  the  implementation  of  well  funded  programs,  such  as  the  U.  S.  Federal 
government's  Star  Schools  Program,  the  vast  possibilities  of  distance  learning  are  being 
increasingly  offered  to  K-12  student  populations  (Schlosser  &  Anderso- ,  1993,  p.  37). 


Many  states  have  or  are  installing  technology  which  will  enable  all  levels  of  education  to  utilize 
distance  education  (Moore.  &  Thompson,  1990;  School  Tech  News.  1986).  This  may  be  good  news  for 
secondary  agricultural  education  programs.  According  to  the  National  Research  Council  (1988),  educators 
must  create  new  ways  to  deliver  agricultural  education  to  a  greater  number  of  students.  The  Iowa 
Communications  Network  may  provide  a  suitable  mechanism  for  delivering  agricultural  instruction  to  a 
larger  audience  of  youth  and  adults. 

Will  distance  education  technology  be  accepted  by  secondary  educators  in  general  and  agricultural 
educators  specifically?  Faculty  resistance  is  often  cited  as  a  major  barrier  to  the  implementation  of  distance 
education  technology  (Dillon  &  Walsh,  1992).  Currently,  few  secondary  educators  have  knowledge  related 
to  distance  education.  And,  faculty  are  seldom  the  subjects  of  research  in  distance  education  (Dillon  et.  al. 
1992). 

Negative  teacher  attitudes,  additional  workloads,  lack  of  funding,  reduced  student  interaction,  lack 
of  time,  fear  of  job  loss,  fear  of  technology,  and  technical  problems  have  all  been  identified  as  obstacles  to 
the  adoption  of  distance  education  technologies  (Bruder,  1989;  Dillon  &  Walsh,  1992;  Hansford  &  Baker, 
1990;  Jackson  &  Bowen,  1993;  Jurasek,  1993;  Koontz,  1989;  Swan  &  Brehmer,  1992).  Teacher 
experience  with  technology  seems  to  be  key  in  overcoming  such  barriers,  however.  Several  researchers 
(Dillon  &  Walsh,  1992;  Jurasek,  1993;  Koontz,  1989)  have  concluded  that  faculty  with  distance  teaching 
experience  generally  have  more  positive  attitudes  toward  technology  mediated  instruction. 

The  transfer  of  technology  from  researcher  to  end  user  is  a  complex  process  and  may  be  useful  to 
researchers  in-understanding  some  teachers'  initial  resistance  to  distance  education  technology.  Five  distinct 
phases  have  been  identified  that  take  place  in  the  adoption  process.  These  phases  are  awareness,  interest, 
evaluation,  trial,  and  adoption  (Lionberger  &  Gwin,  1982;  Rollins,  1993).  At  the  awareness  stage,  the 
teachers  have  heard  about  or  read  about  the  technology,  while  at  the  interest  stage  they  want  to  know  more 
about  the  technology  and  how  it  works.  The  evaluation  stage  involves  the  teacher's  "mental"  decision  to 
try,  or  not  to  try  the  technology.  In  the  trial  stage,  use  of  the  technology  starts  slowly  and  increases  as  the 
teacher  begins  to  appreciate  the  technology.  Once  convinced  of  the  usefulness  of  the  technology,  the  teacher 
enters  the  adoption  phase  and  implements  the  technology. 

Rate  of  movement  from  one  stage  of  the  adoption  process  to  the  next  can  vary  widely,  and  research 
indicates  that  speed  of  adoption  cannot  be  increased  by  skipping  stages  (Lionberger  &  Gwin,  1982).  Other 
considerations  related  to  the  rate  of  adoption  include  teachers'  willingness  to  change,  individual  differences, 
ability  to  understand  the  technology,  and  available  funding. 

Distance  education  technologies  may  be  able  to  help  facilitate  the  modernization  and  improvement 
of  secondary  agricultural  education  programs.  But,  several  questions  must  be  answered  before  this  can 
occur.  What  are  the  attitudes  of  teachers  toward  delivering  instruction  via  interactive  communications 
networks,  and  what  obstacles  might  inhibit  their  use  of  such  systems?  Also,  what  priorities  should  be 
established  for  collaboration  among  existing  agriculture  programs  and  for  course  offerings  to  schools 
without  agriculture  programs? 
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Purpose  and  Objectives 


The  purpose  of  this  descriptive  study  was  to  investigate  the  usefulness  of  an  interactive 
communications  network  for  agricultural  education  at  the  secondary  level.  The  Iowa  Communications 
Network  (ICN)  is  a  2-way  full  motion  fiber  optics  telecommunications  system  capable  of  linking  secondary 
agricultural  education  departments  throughout  Iowa.  The  objectives  of  the  study  were  to: 

1 .  Describe  obstacles  that  may  inhibit  use  of  the  ICN  as  perceived  by  secondary 
agriculture  teachers. 

2.  Describe  secondary  agriculture  teachers'  attitude  toward  using  the  ICN  for 
delivering  agricultural  instruction. 

3 .  Describe  relationships  between  teachers'  attitude,  perceived  obstacles,  and  selected  variables. 

4.  Identify  priorities  for  collaboration  among  secondary  agriculture  programs  in 
delivering  instruction  over  the  ICN. 

5 .  Identify  courses  offered  in  secondary  agriculture  programs  that  are  suitable  or 
unsuitable  for  delivery  via  the  ICN. 

Procedures 

The  population  for  the  study  consisted  of  all  secondary  teachers  of  agricultural  education  in  Iowa 
(N=21 6).  Based  on  Krejcie  and  Morgan's  (1970)  formula  for  a  5%  margin  of  error,  a  random  sample  of  140 
teachers  was  selected  to  participate  in  the  study. 

The  questionnaire  utilized  in  the  study  consisted  of  4  parts  including  the  attitude  toward  the  ICN 
scale,  obstacles  that  may  inhibit  use  of  the  ICN  scale,  questions  related  to  collaboration  and  potential  course 
offerings,  and  selected  demographic  questions.  Content  and  face  validity  were  established  by  a  panel  of 
experts  in  agricultural  education. 

Obstacles  that  may  inhibit  the  use  of  the  ICN  by  secondary  agriculture  teachers  were  identified  by 
interviewing  persons  responsible  for  administering  different  aspects  of  the  ICN,  agriculture  teachers  not 
included  in  the  sample,  and  from  an  instrument  used  by  Swan  (1992)  for  a  similar  purpose  in  North  Dakota. 
Response  categories  for  the  Likert-type  scale  ranged  from  insignificant  (1)  to  significant  (6).  The 
Cronbach's  alpha  reliability  coefficient  for  the  obstacles  scale  was  .82. 

Teachers'  attitude  toward  the  ICN  was  measured  with  a  28  item  Likert-type  scale,  with  five 
response  categories  ranging  from  strongly  disagree  (1)  to  strongly  agree  (5).  The  attitudinal  instrument  was 
tested  for  suitability  and  reliability  with  a  group  of  10  teachers  not  included  in  the  sample.  The  Cronbach's 
alpha  reliability  estimate  was  .93. 

The  questionnaire,  along  with  a  cover  letter  and  a  stamped  return  envelope,  was  sent  to  all 
secondary  agriculture  teachers  included  in  the  sample.  After  10  days,  a  second  mailing  was  sent  to  all 
nonrespondents.  Ten  days  after  the  second  mailing,  a  reminder  letter  was  sent  to  all  nonrespondents 
stressing  the  importance  of  their  participation.  Approximately  10  days  following  the  third  mailing, 
telephone  calls  were  made  to  the  nonrespondents.  One  hundred  and  two  teachers  completed  and  returned  the 
questionnaire  for  a  response  rate  of  73%.  Nonresponse  error  was  controlled  by  comparing  early  to  late 
respondents  (Miller  &  Smith,  1983).  No  significant  differences  were  found  between  early  and  late 
respondents. 

Analysis  of  Data 

All  data  were  analyzed  with  the  SPSS/PC+  personal  computer  program.  Appropriate  statistics  for 
description  (frequencies,  percents,  means,  standard  deviations,  pearson  correlations,  and  point  biserial 
correlations)  were  used.  The  alpha  level  was  set  a  priori  at  .05,  and  Davis'  ( 1 97 1 )  descriptors  were  used  to 
interpret  all  correlation  coefficients. 
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The  agricultural  educators  who  participated  in  the  study  ranged  in  age  from  23  to  64  years.  The 
mean  age  of  respondents  was  36.94  with  a  standard  deviation  of  9.50.  In  regard  to  gender,  90.2%  (92)  of 
the  teachers  were  male. 

Teachers  were  asked  to  report  their  highest  level  of  education.  Bachelors  degrees  were  held  by  7 1  % 
(66)  percent  of  the  respondents,  26.9%  (25)  of  the  teachers  held  masters  degrees,  and  2.2%  (2)  held  doctoral 
degrees.  Teachers  were  also  asked  to  indicate  the  number  of  years  they  had  taught  agricultural  education,  and 
whether  on  not  they  had  tenure.  Years  of  experience  ranged  from  one  to  35  with  a  mean  of  12.44  and  a 
standard  deviation  of  8.51.  Approximately  three-quarters  (77)  of  the  teachers  had  tenure  in  their  current 
positions. 

The  teachers  were  asked  if  their  school  was  currently  connected  to  the  ICN.  They  were  also  asked 
if  they  had  ever  taught  or  taken  a  class  via  the  ICN.  At  the  time  of  the  survey,  22.5%  (22)  of  the  schools 
represented  by  the  agriculture  teachers  were  connected  to  the  ICN.  None  of  the  teachers  had  taught  using 
this  technology.  Nine  teachers  (9.1%)  indicated  that  they  had  taken  at  least  one  course  via  the  ICN. 

The  teachers  responded  to  sixteen  statements  representing  obstacles  which  might  inhibit  their  use 
of  the  ICN.  A  Likert-type  scale  with  response  categories  ranging  from  insignificant  (1)  to  significant  (6) 
was  utilized.  Table  1  shows  that  forty-eight  percent  (49)  of  the  teachers  provided  a  mean  score  in  the  range 
of  4.51  to  5.50  (moderately  significant).  Approximately  39%  (38)  of  the  teachers  reported  mean  scores  in 
the  range  of  3.51-4.50  (slightly  significant).  Means  scores  in  the  range  of  1 .51-3.50  (moderately  or 
slightly  insignificant)  were  reported  by  less  than  eight  percent  (8)  of  the  teachers.  The  overall  mean  score 
for  the  16  obstacles  was  4.49  (slightly  significant),  with  a  standard  deviation  of  .63. 

Table  1 

Overall  Mean  Scores  for  Obstacles  that  May  Inhibit  Use  of  the  Iowa  Communications  Network  by 


Mean 

f 

% 

Cum  % 

1.51-2.50 

4 

1.0 

1.0 

2.51-3.50 

7 

6.8 

7.8 

3.51-4.50 

38 

39.3 

47.1 

4.51-5.50 

49 

48.0 

95.1 

5.51-6.00 

5 

4.9 

100.0 

Total 

102 

100.0 

100.0 

Mean  4.49    Std.  Dev.  .63 

Note:     Based  on  Scale:  1  =  insignificant;  2  =  moderately  insignificant;  3  =  slightly  insignificant;  4  = 
slightly  significant;  5  =  moderately  significant;  6  =  significant 

Table  2  shov/s  the  percentage  of  teachers  who  selected  slightly  significant,  moderately  significant, 
or  significant  for  each  of  the  sixteen  obstacles.  School  and  class  scheduling  problems  were  considered  most 
significant  by  the  agricultural  educators.  Lack  of  local  support  staff,  the  inability  to  have  lab  sessions,  and 
materials  distribution  were  each  considered  slightly  significant,  moderately  significant,  or  significant  by 
87.3%  of  the  teachers.  Costs,  lack  of  training,  and  preparation  time  were  considered  slightly  significant  to 
significant  obstacles  by  80-85%  of  the  agriculture  teachers.  Obstacles  receiving  the  lowest  frequencies  in 
the  slightly  significant,  moderately  significant,  or  significant  categories  were  lack  of  student  interest  and 
negative  attitudes  of  teachers  towards  the  ICN. 
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Table  2 

Percentage  of  Teachers  Who  Selected  Slightly  Significant.  Moderately  Significant,  or  Significant  for  Each 
Obstacle 


Obstacle  % 


1 

1  . 

K,  UUIU  IlltUlUIl  Ul  5vllt,*JUlC5  UCIWCC1I  IVJVlJ. 

94.1 

Z. 

ine  iv^rN  couiu  crcaie  scncuuiing  ptuuicni&. 

88.2 

i 

j . 

LaDoraiory  sessions  cdnnui  dc  laugm  via  ine  iv^i><. 

87.3 

A 

4. 

uisiriouting  materials  oeiween  Mies. 

87.3 

c 

J . 

T  o/^l'  r\l  lrv*nl  cnr\r\ort  ctflTT 
LaCK  Ul  luval  cjuppuil  blall. 

87.3 

0. 

^unprviQ^H  norimltiirjil  exneripnces  cannot  he 

86.3 

managed  via  the  ICN. 

85.3 

7 

t^osis  associated  wnn  umii£  ine  i^in. 

8. 

Lack  of  training. 

83.3 

o 

Preparation  time  needed  by  teachers. 

82.4 

10. 

Fear  that  the  ICN  would  reduce  the  number 

78.4 

of  agriculture  programs. 

77.5 

11. 

Agriculture  teachers  are  to  busy  to  teach  via 

the  ICN. 

12. 

Lack  of  incentives  for  teaching  via  the  ICN. 

77.5 

13. 

Administrators  do  not  understand  teachers'  needs 

77.5 

when  teaching  via  the  ICN. 

68.6 

14. 

Difficulty  in  establishing  cooperative  relationships 

among  schools. 

15. 

Negative  attitude  of  teachers  towards  the  ICN. 

61.8 

16. 

Lack  of  student  interest. 

58.8 

On  a  five-point  Likert-type  scale,  teachers  were  asked  to  respond  to  28  statements  related  to  their 
attitude  toward  the  use  of  the  ICN  to  teach  agriculture.  Table  3  shows  that  62.7%  (64)  of  the  teachers 
provided  a  mean  score  in  the  range  of  2.51  to  3.50  (undecided).  An  additional  32%  (33)  of  the  agriculture 
teachers  provided  a  mean  score  in  the  range  of  3.5 1-4.50  (agree).  The  remaining  4.9%  (5)  of  the  teachers 
provided  mean  scores  between  1 .51  and  2.50  (disagree).  The  overall  mean  score  for  the  28  attitudinal 
statements  was  3.26  (undecided)  with  a  standard  deviation  of  .47. 


Table  3 

Overall  Mean  Scores  for  Agriculture  Teachers'  Attitude  Toward  Using  the  Iowa  Communications  Network 
to  Teach  Agriculture 


Mean 

f 

% 

Cum  % 

1.51-2.50 

5 

4.9 

4.9 

2.51-3.50 

64 

62.7 

67.6 

3.51-4.50 

33 

32.4 

100.0 

Total 

102 

100.0 

100.0 

Mean  3.26     Std.  Dev.  .47 

Note:     Based  on  Scale:  1  = 

strongly  disagree;  2=  disagree;  3=  undecided;  4= 

agree;  5=strongly  agree 

Pearson  correlations  and  point  biserial  correlations  were  used  to  describe  relationships  between 
obstacles  that  may  inhibit  the  use  of  the  ICN  and  selected  variables  (Table  4).  The  associations  ranged  in 
magnitude  from  negligible  to  moderate.  Teachers  who  provided  higher  scores  on  the  obstacles  scale  tended 
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to  have  less  positive  attitudes  towards  the  ICN,  were  less  likely  to  be  located  in  a  school  connected  to  the 
ICN,  and  were  younger.  The  association  between  years  of  teaching  experience  and  perceived  significance  of 
the  obstacles  was  negligible. 


Table  4 

Summary  of  Relationships  Between  Obstacles  That  May  Inhibit  Use  of  the  ICN  and  Selected  Variables 


Variable 

Association 

Attitude  toward  ICN 

-.36* 

School  connected  to  ICN 

-.13 

Years  of  teaching  experience 

-.08 

Age 

-.16 

*  p  =  >.05 


Table  5  shows  the  associations  between  attitude  toward  using  the  ICN  for  delivering  agricultural 
instruction  and  selected  variables.  The  associations  ranged  in  magnitude  from  negligible  to  low.  Female 
agriculture  teachers  tended  to  have  more  positive  attitudes  towards  using  the  ICN  to  teach  agriculture.  The 
association  between  years  of  teaching  experience,  connection  to  the  ICN,  and  age  were  negligible. 


Table  5 

Summary  of  Relationships  Between  Attitude  Toward  the  ICN  and  Selected  Variables 


Variable  Association 


School  connected  to  ICN  .06 

Years  of  teaching  experience  -.07 

Gender  -21* 

Age  .01 


*  p  =  >.05 


Agriculture  teachers  were  asked  to  list  units  of  instruction  that  they  would  like  to  receive  from  other 
agriculture  programs  through  the  ICN.  A  total  of  275  units  of  instruction  were  listed  by  the  102 
agriculture  teachers  participating  in  the  study.  Units  of  instruction  were  placed  into  12  content-related 
categories  by  the  researchers.  Table  6  shows  that  units  related  to  agricultural  economics  (25.8%)  were  listed 
most  frequently  as  priority  units  for  reception.  Horticulture,  floriculture,  and  landscaping  units  (13.8%) 
were  the  second  most  frequently  cited  units  followed  by  animal  sciences  (11 .2%),  agronomy  (9.5%), 
aquaculture  (9.5%),  agricultural  mechanics  (8.3%),  and  biotechnology  (5.5%).  Categories  representing  less 
than  5%  of  the  total  number  of  units  included  natural  resources  and  the  environment,  careers  in  agriculture, 
computers,  and  leadership,  FFA  and  SAE.  Approximately  6%  of  the  units  were  grouped  into  a 
miscellaneous  category  and  included  such  units  as  forestry,  food  technology,  international  agriculture,  and 
agricultural  journalism. 

Agriculture  teachers  were  also  asked  to  list  units  of  instruction  that  they  would  be  willing  to  teach  via 
the  ICN.  A  total  of  164  units  of  instruction  were  listed  by  the  102  agriculture  teachers  who  participated  in 
the  study.  Table  6  shows  that  units  related  to  animal  sciences  (25.6%)  were  listed  most  frequently  as 
priority  units  for  delivery.  The  second  most  frequently  cited  category  was  agricultural  economics  (23.2%) 
and  was  followed  by  agronomy  (1 1.6%),  horticuhure,  floriculture,  and  landscaping  (8.5%),  agricultural 
mechanics  (7.3%),  and  leadership,  FFA,  and  SAE  (5.5%).  Categories  representing  less  than  5%  of  the  total 
number  of  units  included  natural  resources  and  the  environment,  computers,  careers  in  agriculture, 
biotechnology,  aquaculture,  and  the  miscellaneous  category. 


Due  to  the  nature  of  the  instruction  in  secondary  agriculture  programs,  it  could  be  hypothesized  that 
only  select  course  offerings  are  suitable  for  delivery  via  the  ICN.  Agriculture  teachers  who  participated  in 
the  study  were  asked  to  list  titles  of  courses  (semester  or  year-long)  that  could  be  delivered  via  the  ICN  to 
schools  without  an  agriculture  teacher.  A  total  of  210  course  titles  were  listed  by  the  agriculture  teachers. 
The  researchers  collapsed  the  course  titles  into  nine  categories  which  are  presented  in  Table  7.  Course  titles 
related  to  agricultural  economics  (35.2%)  were  listed  most  often  as  courses  that  were  suitable  for  delivery 
via  ICN.  The  second  most  frequently  cited  category  of  course  titles  was  agronomy  (19.5%)  arid  was 
followed  by  animal  sciences  (18.6%),  horticulture,  floriculture,  and  landscaping  (5.7%),  and  natural 
resources  and  the  environment  (5.2%).  Categories  representing  less  than  5%  of  the  course  titles  included 
agricultural  mechanics,  leadership,  FFA,  and  SAE,  and  aquaculture.  Approximately  1 1%  of  the  course 
titles  were  grouped  into  a  miscellaneous  category  and  included  such  titles  as  agricultural  communications, 
Iowa  agriculture,  agricultural  issues,  and  agricultural  chemicals. 

Table  6 

Categories  of  Priority  Units  of  Instruction  that  Agriculture  Teachers  Desire  to  Receive  or  Would  be  Willing 
to  Deliver  via  the  ICN 


Receive  Deliver 


Unit 

f 

% 

f 

% 

Agricultural  Economics 

71 

25.8 

38 

23.2 

Horticulture/Floriculture/Landscaping 

38 

13.8 

14 

8.5 

Animal  Sciences 

31 

1 1.2 

42 

25.6 

Agronomy 

26 

9.5 

19 

11.6 

Aquaculture 

26 

9.5 

4 

2.4 

Agricultural  Mechanics 

23 

8.3 

12 

7.3 

Biotechnology 

15 

5.5 

4 

2.4 

Natural  Resources/Environment 

13 

4.7 

6 

3.7 

Careers  in  Agriculture 

6 

2.2 

4 

2.4 

Computers 

5 

1.8 

5 

3.1 

Leadership/FFA/SAE 

4 

1.5 

9 

5.5 

Miscellaneous 

17 

6.2 

7 

4.3 

Total 

275 

100.0 

164 

100.0 

Table  7 

Categories  of  Priority  Courses  That  Could  be  Offered  via  the  ICN  to  Schools  with 

no  Agriculture  Teacher 

Suitable 

Not  Suitable 

Course 

f 

% 

f 

% 

Agricultural  Economics 

74 

35.2 

9 

4.8 

Agronomy 

41 

19.5 

19 

10.2 

Animal  Sciences 

39 

18.6 

12 

6.5 

Horticulture/Floriculture/Landscaping 

12 

5.7 

30 

16.1 

Natural  Resources/Environment 

1 1 

5.2 

5 

2.7 

Agricultural  Mechanics 

6 

2.9 

88 

47.3 

Leadership/FFA/SAE 

3 

1.4 

8 

4.3 

Aquaculture 

1 

.5 

5 

2.7 

Miscellaneous 

23 

1 1.0 

10 

5.4 

Total 

210 

100.0 

186 

100.0 
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Agriculture  teachers  were  also  asked  to  list  titles  of  courses  (semester  or  year-long)  that  would  be 
unsuitable  for  ICN  delivery.  A  total  of  186  course  titles  were  listed  by  the  agriculture  teachers.  Table  7 
shows  that  agricultural  mechanics  courses  (47.3%)  were  most  frequently  cited  as  not  suitable  for  ICN 
delivery  followed  by  horticulture,  floriculture,  and  landscaping  courses  (16.1%),  agronomy  courses  (10.2%), 
animal  science  courses  (6.5%),  and  miscellaneous  course  titles  (5.4%).  Categories  representing  less  than 
5%  of  the  course  titles  included  agricultural  economics,  natural  resources  and  the  environment,  leadership, 
FFA,  and  SAE,  and  aquaculture. 

Conclusions  and/or  Recommendations 

Overall,  the  16  obstacles  to  using  the  ICN  in  secondary  agriculture  programs  were  perceived  to  be 
slightly  significant.  Teachers  were  most  concerned  with  scheduling  problems,  but  were  also  concerned  that 
laboratory  sessions  and  supervised  agricultural  experience  programs  could  not  be  managed  over  the  system. 
Additionally,  the  respondents  were  concerned  with  costs,  lack  of  training,  and  incentives  for  using  the 
system. 

Perhaps  scheduling,  training,  and  incentives  are  less  problematic  than  concerns  related  to  supervised 
agricultural  experiences  and  laboratory  experiences.  Can  quality  programs  in  agricultural  education  be 
delivered  while  sacrificing  the  application  of  learning  provided  through  supervised  agricultural  experiences 
and  laboratory  experiences?  Do  agriculture  teachers  really  have  to  sacrifice  these  components  of  an 
agriculture  program?  It  was  recommended  that  pilot  or  demonstration  programs  be  developed  that  include 
laboratory  and  hands-on  learning  experiences  within  the  interactive  distance  education  delivery  mechanism. 
The  interactive  and  video  components  of  distance  education  should  be  exploited  to  demonstrate  viable 
alternatives  to  conventional  methods  of  teaching  agricultural  education. 

Data  suggest  that  secondary  agriculture  teachers  are  undecided  about  using  Uie  ICN  as  a  tool  for 
teaching  agriculture.  If  attitudes  are  a  reflection  of  an  individual's  personal  perspective  and  are  strongly 
predictive  of  behavior  (Na  and  Lee,  1993),  what  does  this  tell  us  about  agriculture  teachers'  willingness  to 
use  this  educational  technology?  Perhaps  Lionberger  et  al.'s  (1982)  adoption  process  theory  could  explain 
the  current  situation.  The  ICN  was  connected  to  less  than  25%  of  the  schools  represented  in  this  study. 
This  might  indicate  that  most  of  the  secondary  agriculture  teachers  were  at  the  early  stages  of  the  adoption 
process.  Maybe  the  teachers'  indecisiveness  about  using  the  ICN  was  related  to  their  status  in  the  adoption 
process.  It  was  recommended  that  teacher  educators  provide  secondary  agriculture  teachers  with  current 
information  related  to  the  ICN  to  increase  awareness  and  stimulate  interest.  Also,  secondary  agricultural 
education  teachers  should  be  provided  opportunities,  both  as  a  recipient  and  provider  of  distance  education, 
to  gain  experience  with  ICN  technology.  Studies  in  technology  have  shown  that  teacher  attitudes  become 
more  positive  as  a  result  of  experience  with  technology  (Na,  S.  &  Lee,  M.,  1993;  Rollins,  1993). 

The  highest  priority  for  collaborative  efforts  among  schools  with  agriculture  programs  were  in  the  areas 
of  agricultural  economics  and  horticulture.  Teachers  also  cited  units  of  instruction  (aquaculture  and 
biotechnology)  that  are  related  to  current  curriculum  initiatives  in  agricultural  education  as  priorities  for 
collaboration.  Teacher  educators  should  plan,  organize,  and  deliver  inservice  education  for  agriculture 
teachers  in  curriculum  development  and  strategies  for  lesson  presentation  particularly  for  agricultural 
economics  and  horticulture  related  units.  The  data  suggest  an  adequate  number  of  teachers  are  willing  to 
teach  units  of  instruction  in  the  priority  areas  via  the  ICN.  Teacher  educators  should  promote  the 
involvement  of  secondary  agricultural  education  teachers  in  using  the  system  to  improve  agriculture 
curriculum  in  secondary  agriculture  programs. 

Interestingly,  different  teachers  perceived  the  same  content-related  categories  of  courses  to  be  both 
suitable  and  unsuitable  for  delivery  via  the  ICN  to  schools  with  no  agriculture  teacher.  A  clear  pattern  was 
evident  regarding  the  suitability  of  agricultural  mechanics  courses  and  agricultural  economics  courses. 
Teachers  generally  agreed  that  agricultural  mechanics  courses  were  not  suited  to  ICN  delivery,  but 
agricultural  economics  courses  were  suited  to  ICN  delivery.  Teacher  educators,  secondary  agriculture 
teachers,  administrators  and  others  with  an  interest  in  agriculture  should  work  collaboratively  to  facilitate 
the  delivery  of  instruction  in  and  about  agriculture  to  schools  without  agriculture  teachers.  The  teachers 
who  participated  in  this  study  placed  considerable  emphasis  on  agricultural  economics,  but  agronomy  and 
animal  sciences  courses  were  also  listed  as  promising  areas  for  course  delivery. 
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Introduction 


Who  are  we?  Where  are  we  going?  What  useful  purpose  do  we  serve?  Such  introspective 
questions— although  viewed  as  pointless  by  some— are  necessary  questions  for  a  profession  to  ask  of  itself 
periodically.  During  the  late  1980s  the  board  of  directors  of  the  Association  for  Educational 
Communications  and  Technology  (AECT)  undertook  strategic  planning  for  the  future  of  association.  They 
began  by  looking  at  the  members  who  constitute  the  organization  and  who  populate  the  educational  media 
and  technology  (em/t)  profession.  As  they  did  so,  they  realized  that  there  are  many  questions  about  "who 
we  are"  and  "where  we  are  going"  that  could  not  be  answered  with  any  degree  of  certainty.  The  data  either 
didn't  exist  or  weren't  readily  accessible. 

This  dearth  of  information  about  the  profession  was  considered  serious  enough  to  set  up  a  task 
force,  later  a  committee,  on  "Research  on  the  Profession."  This  committee,  chaired  first  by  George  Grimes 
and  later  by  Michael  Molenda,  operated  from  1990  to  1994  with  the  charge  of  doing  something  to  remedy 
this  information  gap.  The  group  decided  that  a  long-term  solution  must  include  the  promotion  of 
continuing  research  about  the  educational  media  and  technology  profession  and  the  establishment  of  a 
mechanism  for  making  this  research  available  to  those  seeking  information. 

Fortunately,  the  committee  discovered  early  that  Donald  P.  Ely,  director  of  the  ERIC 
Clearinghouse  on  Information  and  Technology,  had  already  committed  the  Clearinghouse  to  serving  the 
storage  and  dissemination  functions  envisioned  by  the  committee.  From  that  point  the  committee  focused 
on  a  strategy  of: 

-staking  out  the  territory  to  be  included  in  "research  on  the  profession," 

-determining  where  the  knowledge  gaps  are,  and 

-encouraging  further  research  to  fill  those  knowledge  gaps. 
This  report  is  the  first  effort  to  stake  out  the  territory  and  explore  the  existing  knowledge  base  within  that 
territory.  It  is  the  hope  of  the  committee  that  this  Agenda  will  stimulate  interest  among  researchers  and 
apprentice  researchers.  It  comprises  a  set  of  questions,  the  answers  to  which  cast  light  on  who  we  are, 
where  we  are  going,  and  what  useful  purpose  we  serve. 

A  preliminary  draft  of  this  paper  was  presented  at  the  1995  AECT  annual  convention.  Reactions 
were  given  by  Donald  Ely,  Sandy  Patton,  and  Philip  Doughty.  Each  made  valuable  substantive 
suggestions  which  have  been  incorporated  into  this  "Proceedings"  version. 

The  Agenda 

I.  Definition  of  the  field 

A.  Boundaries:  What  is  the  currently  accepted  definition  of  the  educational  media  and  technology 
(em/t)  field? 

0 within  the  profession 

°among  outside  individuals  and  groups 

B.  Image:  How  is  the  field  perceived? 

"within  the  profession 

"among  outside  individuals  and  groups 

C.  Historical  Perspective:  How  have  perceptions  changed  over  time? 

D.  Justification  for  existence:  Are  em/t  specialists  needed?  What  valuable  functions  do  they 
serve. ..within  organization?.. .within  society?  What  potential  harm  might  be  caused  by  ignorant, 
incompetent,  or  unethical  practice?  Are  employers  well  informed  about  qualifications  em/t 
professionals  should  possess? 


II.  Positions  held 

A.  Position  Titles:  What  positions  are  currently  held  by  em/t  professionals 

...in  schools? 

...in  higher  education? 

...in  business/industry? 

...in  government? 

...other? 

B.  Marketplace:  What  sorts  of  positions  are  currently  being  advertised? 

C.  Qualifications:  What  entry  qualifications  are  expected  by  employers? 

D.  Marketplace  Status:  Is  there  a  balance  between  supply  and  demand  of  graduates? 

E.  Roles:  How  are  the  roles  of  em/t  specialists  defined? 

0  What  different  roles  are  defined  and  advocated  within  the  field? 
0  What  different  roles  are  actually  being  played  by  practitioners? 
°  What  do  exemplary  performers  do  that  makes  them  exemplary? 

F.  Career  Paths:  What  are  typical  career  paths  followed  by  em/t  professionals? 

G.  Position/Role  Trends:  Is  there  a  pattern  of  change  in  the  types  of  positions  available,  the 
supply  of  qualified  applicants,  the  roles  played,  or  career  paths? 

III.  Compensations  vs.  Demands 

A.  Compensation  Status:  What  are  the  current  salary  ranges  for  different  positions?  Other 
compensations?  ' 

B.  Compensation  Variables:  Do  salaries  vary  by  educational  level?  Geographical  location? 
Workplace?  Gender?  Other  variables? 

C.  Compensation  Trends:  Is  there  a  pattern  of  change  in  salaries  or  other  compensations? 

D.  Job  Demands:  What  responsibilities  do  em/t  professionals  have?  What  special  stresses  are 
they  subjected  to? 

E.  Job  Demand  Trends:  How  are  job  demands  changing?  How  are  stresses  changing? 

IV.  Professional  Preparation  and  Standards 

A.  Academic  Study  Opportunities:  Where  are  degrees  in  em/t  offered? 

"Master's 

"Specialist/Certificate 
"Doctoral 

B.  Distribution  of  Students:  How  many  students  graduate  from  each  program/degree? 

C.  Academic  Curricula:  What  are  the  curricula  offered  in  these  programs?  How  do  they  differ? 

D.  Core  Curriculum:  To  what  extent  is  there  agreement  on  a  "core  curriculum?" 


E.  Program  Quality:  Are  there  evaluative  data  about  these  programs?  (e.g.  relative  to  relevance  of 
curricula  to  career  preparedness) 

F.  Preparation/Practice  Match:  What  sorts  of  training  are  most  valuable  in  helping  students 
become  successful  practitioners? 

G.  Academic  Program  Trends:  Are  the  number  or  type  of  programs  or  graduates  changing?  Are  the 
curricula  changing?  Are  the  programs  flourishing,  declining,  remaining  steady? 

H.  Continuing  Professional  Development:  What  sorts  of  continuing  education  are  needed? 
Wanted? 

I.  Standards:  What  standards  of  professional  competence  or  ethical  codes  are  advocated?  Enforced? 

V.  Communications 

A.  Communications  among  professionals:  How  do  professionals  communicate  among 
themselves? 

°  adequacy  of  journals  and  periodicals 

°  accessibility  of  publications 

B.  Information  sources:  How  is  needed  information  stored  and  retrieved? 

°  accessibility  of  print  and  electronic  databases 

C.  Linkages:  How  do  professionals,  individually  and  in  groups,  connect  with  others? 

°  on  a  local  and  regional  basis? 
°  on  a  national  basis? 

°  on  an  international  basis? 

VI.  Professional  Organizations 

A.  Membership:  Which  professional  organizations  do  em/t  professionals  belong  to? 

°national  level 
°state  level 
°local  level 

B.  Membership  Overlap:  To  what  degree  is  there  overlap  in  membership  among  associations? 

C.  Association  Functions:  What  sorts  of  services  are  offered  or  activities  are  undertaken  by  these 
associations? 

D.  Association  Leadership:  Who  hold  leadership  positions  in  these  associations?  How  are 
positions  of  leadership  gained?  What  provision  is  made  for  leadership  development? 

H.  Association  Trends:  Any  change  in  the  pattern  of  association  membership?  How  are  internal 
and  external  forces  affecting  the  growth  and  health  of  professional  organizations? 
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VII.  The  Workplace 


A.  Workplace  Settings:  In  what  organizational  settings  do  em/t  professionals  work?  (e.g.  school 
or  college  media  centers,  local  or  state  school  agencies,  university  academic  departments, 
corporations,  etc.) 

B.  Workplace  Satisfaction:  Do  em/t  professionals  experience  satisfaction  with  their  workplace 
environment? 

C.  Job  Satisfaction:  Do  em/t  professionals  experience  satisfaction  in  their  jobs? 

D.  Workplace  Trends:  Are  these  organizational  settings  declining  or  growing  in  number?  Are 
they  declining  or  growing  in  financial  support?  Are  roles  changing?  In  what  ways? 


VII.  External  forces 

A.  Societal  Forces:  What  societal  forces  are  affecting  the  profe^ion?  What  are  the  forces  that 
assist  or  impede  the  appropriate  adoption  of  technology  in  education? 

B.  Governance  Forces:  What  changes  are  talcing  place  in  the  governance  of  public  K-12  education? 
Public  and  private  higher  education?  How  are  these  governance  changes  affecting  the  em/t 
profession?  How  do  these  forces  aid  or  impede  the  appropriate  adoption  of  technology? 

C.  Workplace  Forces:  What  workplace  forces  are  affecting  the  profession? 

D.  Technological  Forces:  How  are  technological  forces  affecting  the  profession?  How  does 
change  in  technology  itself  affect  the  adoption  or  rejection  of  technology  in  education  and  training? 

E.  Historical  Patterns  and  Trends:  Historically,  what  has  been  the  impact  of  various  external 
forces  on  th«;  size  of,  shape  of,  or  directions  taken  by  the  field? 

The  Knowledge  Base 

We  see  two  steps  being  critical  to  establishing  the  knowledge  base  of  the  em/t  field:  first, 
surveying  the  literature  on  the  profession  of  educational  technology,  and  second,  assessing  the  literature  for 
adequacy,  gaps,  obsolescence,  and  so  on.  We  are  reporting  the  first  step  here  in  the  form  of  a  selected 
bibliography  of  works  about  this  profession.  It  is  based  on  a  larger  database-of  some  495  citations—that  is 
published  separately  by  Educational  Technology  Publications  as  The  Educational  Technology  Profession:  A 
Bibliographic  Overview  of  a  Profession  in  Search  of  Itself  by  J.  Fred  Olive;  edited  by  the  series  editor, 
William  D.  Milheim. 

The  second  step  entails  a  critical  analysis  of  the  database,  assessing  what  is  known  for  its 
completeness  and  up-to-date-ness  and  looking  for  gaps  in  the  knowledge  base.  This  h=  the  larger  task  thr.» 
still  remains.  We  invite  other  scholars  to  join  in  this  endeavor. 
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MULTIMEDIA 


Educators  have  used  the  term  multimedia  in  a  global  sense  for  years:  outside  the  classroom  the  term 
was  rarely  heard.  Research  on  components  of  multimedia  was  spurred  after  the  second  World  War  by  the 
military  interested  in  increasing  training  effectiveness  and  conducted  by  new  research  graduates  eager  to  carry 
out  the  studies.  For  example,  a  research  team  was  formed  at  Penn  State  which  for  years  investigated  the 
effects  of  movies  on  learning  outcomes.  Later  the  thrust  moved  to  video  and  through  the  work  of  Dwyer 
and  others  has  concentrated  on  the  roles  of  graphics  and  color  in  learning. 

In  recent  years,  the  prospects  of  huge  businesses  to  be  spun  out  of  the  multimedia  and  information 
superhighway  efforts  has  brought  the  term  multimedia  out  of  the  closet  and  into  the  public  view.  For 
example,  Naisbitt  (1994)  in  his  newest  look  to  the  future  has  written  that  multimedia  will  become  a 
dominant  force  in  the  next  decade  and  the  two  most  significant  applications  will  consist  of  entertainment 
and  education.  On  the  communications-side  of  things,  software  giant  Microsoft  and  cellular  phone  giant 
McCaw  recently  announced  plans  to  blanket  the  earth  with  communcations  satellite  to  make  electronic 
communication  available  anywhere  in  the  world.  The  potential  for  such  systems  to  impact  education  and 
training  is  enormous,  but  what  does  the  research  tells  us  about  the  field. 

For  now  multimedia  may  be  defined  as  the  components  of  conventional  media  (computer,  video, 
graphics,  animation,  audio,  color)  along  with  their  path  to  complete  integration,  which  is  now  just  in  its 
infancy. 

On  the  assumption  that  understanding  and  directing  the  formation  of  the  whole  can  be  increased  by 
*     examining  its  parts,  this  paper  will  examines  the  multimedia  component  of  animation.  Animation  is 
examined  as  it  is  the  new  kid  on  the  block  and  which  will  likely  grow  quickly  as  the  hardware  and  software 
systems  which  brought  us  desktop  publishing  and  desktop  video  will  soon  bring  desktop  animation  to 
educators  and  trainers. 


GRAPHICS  AND  ANIMATION  IN  INSTRUCTION 


Research  Findings 


Graphics  in  instruction  is  defined  as  any  representation  of  an  object,  concept  or  process,  as  perceived 
through  the  eye,  which  does  not  rely  on  the  use  of  text  or  numbers.  Animation  refers  to  the  use  of  a  series 
of  graphics  which  change  over  time. 

Extensive  research  on  graphics  occured  during  the  1960's  and  70's  and  has  diminished  in  recent  years. 
Subsequent  to  these  studies  which  support  the  positive  effects  of  graphics  on  achievement,  the  computer 
has  become  the  dominant  mode  of  instruction  in  education.  Apparently  researchers  believe  the  findings  and 
conclusions  will  continue  to  be  valid  with  graphics  delivered  by  computer  based  instruction  (CBI).  This 
argument  makes  the  plausible  a  sumption  that  the  type  of  feature  represented  in  instruction  is  more 
important  than  the  number  of  different  ways  it  is  represented. 

It  was  found  that  graphics  and  visuals  increase  the  amount  learned  by  adults  (Alesandrini,  1984)  and  by 
children  (Pressley,  1977).  Alesandrini  and  Rigney  (1981)  found  graphics  to  be  an  effective  review  strategy 
compared  with  verbal  strategies.  In  a  study  which  examined  student  attitude  toward  graphics-based  learning, 
Rigney  and  Lutz  (1976)  concluded  that  using  graphics  as  analogies  in  CBI  resulted  in  high  levels  of 
satisfaction  with  the  learning  experience. 

Willows  (1978)  was  concerned  about  potential  interference  between  the  messages  provided  by  text  and 
graphics  in  textbooks.  Unlike  texts,  CBI,  permits  control  over  the  presentation  to  avoid  such  conflicts. 
For  example,  turning  a  page  of  a  book  may  reveal  a  graphic  and  text,  but  with  computer  control,  the 
graphic  can  be  presented  first  and  the  text  held  until  a  certain  time  had  expired,  or  vice  versa. 

Dwyer  (1970)  showed  that  simple  line  drawing  graphics  tend  to  be  superior  to  photographs  or  other 
more  realistic  drawings.  The  key  seems  to  be  the  relevance  of  the  cues  to  the  learning  task.  For  example, 
using  a  photograph  of  a  car  engine  to  teach  about  the  location  of  the  carbeurator  might  be  appropriate  in 
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terms  of  relevant  cues,  whereas  the  same  photograph  would  be  inappropriate  to  teach  about  the  structure  and 
function  of  the  carb  itself. 

Joseph  and  Dwyer  (1982)  concluded  that  the  integration  of  realistic  and  abstract  graphics  may  reduce 
achievement  differences  between  students  of  different  ability  levels.  This  appears  to  interact  with  the 
concreteness  or  abstractness  of  the  topic  to  be  learned.  In  teaching  about  computers,  for  example,  the  parts 
of  a  computer  can  be  highly  concrete  and  easy  to  represent  graphically,  while  the  functions  of  a  computer 
are  quite  abstract  and  therefore  not  as  conveniently  represented. 

Rigney  and  Lutz  (1976)  elaborated  effects  of  graphics  in  instruction  by  urging  students  to  form  mental 
images  as  they  studied  the  material.  Increased  learning  resulted  from  this  combination.  These  results  are 
supported  by  the  research  of  Canelos  (1979)  who  showed  that  training  in  visual  imagery  techniques  resulted 
in  better  learning  on  a  highly  visual  but  real-world  memory  task. 

Another  factor  about  graphics  needs  to  be  considered  for  both  research  and  learner  evaluation.  The 
amount  learned  from  instruction  using  graphics  can  be  suppressed  and  appear  not  to  have  been  learned  if  the 
examination  does  not  contain  the  graphics  stimuli  that  were  present  during  learning.  Szabo.  DeMelo  & 
Dwyer  (1981)  showed  that  achievement  scores  were  significantly  higher  when  the  graphics  used  during 
instruction  were  incorporated  into  the  testing  protocol. 

Animation  does  not  have  the  rich  research  history  that  is  associated  with  color  and  graphics.  This  may 
be  due  to  the  fact  that  until  recently,  creation  of  animation  required  mainframe  computers  with  complex 
software  and  high  skill  levels.  Today,  basic  animations  can  be  created  using  micros  and  relatively 
inexpensive  and  easy  to  use  software.  Furthermore,  today's  computers  are  capable  of  high  resolutions  of  72 
pixels  per  square  inch,  24  bit  color  (14M  colors),  and  30  frames  per  second  video.  While  the  realism 
provided  by  the  new  technology  seductively  implies  increased  learning  effectiveness,  prior  research  results 
showing  the  superiority  of  simple  graphics  should  make  us  cautious  about  assuming  effectiveness. 

In  an  early  study  using  movies  to  provide  animation,  motion  was  shown  to  be  superior  to  slides  or 
sequential  photos  in  presenting  time-  and  motion-based  concepts  (Wells,  1973).  The  reverse  was  true  for 
concepts  involving  the  presentation  of  spatial  entities. 

Baek  and  Layne  (1988)  compared  learning  conditions  of  text  only,  text  plus  graphics,  and  text  plus 
animation.  The  adults  in  the  study  scored  higher  in  the  animation  condition  than  either  text  or  graphics. 
The  animation  condition  also  resulted  in  less  study  time,  suggesting  that  animation  results  in  more  efficient 
learning.  In  another  study  with  adult  learners,  Mayton  (1991)  found  increased  scores  in  the  animation 
condition  immediately  after  study  persisted  and  were  measurable  one  week  later. 

Rieber  and  Boyce  (1990)  compared  animation-based  instruction  with  carefully  designed  verbal 
presentations  which  used  highly  imaginative  examples  and  illustrations.  The  results  with  an  adult  sample 
indicated  no  significant  difference  in  the  amount  learned  but  the  animation  group  required  less  time  to 
retrieve  the  information  they  learned.  One  would  have  to  ask  whether  the  results  would  follow  with 
children  who  may  not  have  sufficient  experience  bases  to  accurately  imagine  the  examples  and  illustrations 
used. 

It  remains  to  be  seen  whether  the  gains  in  learning  or  reductions  in  learning  time  that  animation  seems 
to  offer  are  worth  the  added  time  and  energy  needed  to  create  animations,  compared  with  other  forms  of 
instruction  such  as  mental  imagery  and  visual  elaboration. 

A  RECENT  STUDY  COMPARING  ANIMATION,  GRAPHICS  AND  TEXT 
Goal  of  the  Study 

The  authors  recently  completed  a  comparison  of  animation,  still  graphics,  and  text  only  on  the 
acquisition  of  the  mathematics  skill  of  using  a  compass  to  create  triangles  and  attitude  toward  the 
instruction  (Poohkay.  1994).  It  was  hypothesized  that  animation  would  result  in  greater  achievement  of  the 
learning  task,  followed  by  graphics,  and  lastly  text  would  yield  the  lowest  achievement.   The  same 


hypotheses  were  posed  for  attitude  toward  mathematics  instruction  by  computer.  The  instruction  was 
delivered  by  computer  based  instruction. 

Materials  and  Methods 

The  instruction  used  a  single  lesson  from  a  larger,  grade  10  mathematics  curriculum  designed  for 
distance  delivery.  The  objective  of  the  lesson  was  to  be  able  to  use  a  compass  to  create  triangles  from 
given  line  segments.  Three  versions  of  the  lesson  were  developed  and  differed  only  in  their  use  of  graphics. 
The  animation  lesson  included  several  animations  of  the  process  of  constructing  triangles  with  a  compass. 
The  graphics  version  replaced  the  animations  with  a  series  of  static  graphics.  The  text-only  version  used  no 
graphics  at  all.  See  Figures  1  and  2  below  for  the  graphics  and  text  versions,  respectively.  (The  animation 
sequence  will  be  demonstrated  as  part  of  the  formal  presentation  of  this  paper) 

Lessons  were  developed  using  Authorware  Professional  for  the  Macintosh  and  delivered  on  a  network  of 
Macintosh  Ilci  s.  The  sample  of  147  volunteers  was  drawn  from  a  group  of  undergraduate  elementary 
education  majors  who  were  completing  a  required  course  in  elementary  mathematics  teaching  methods. 

One  hundred  percent  of  those  approached  agreed  to  participate  in  the  study  which  was  conducted  during 
one  scheduled  class  period.  Upon  entering  the  computer  lab,  students  completed  an  informed  consent  form 
and  a  mathematics  skill  pretest.  They  were  then  randomly  assigned  to  the  instruction  using  one  of  the  three 
instructional  methods  noted  above.  As  they  completed  the  lesson,  they  were  tested  on  their  ability  to  draw 
several  triangles  using  compasses  provided  by  the  researchers.  Finally,  they  completed  a  Likert-type  scale 
which  assessed  attitude  toward  mathematics  instruction  by  computer. 


Figure  1.  Example  of  Graphics  Version  of  Triangle  Construction  Lesson 
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When  constructing  a  building 
certain  eQuiement  and  tools  ere 
needed. 

Some  of  the  tools  used  in 
constructing  a  house  are 
hammers,  saws,  and  drills. 


When  constructing  triangles  you 
are  allowed  only  a  eomssss  and  a 
straightedge. 

Using  the  above  listed  tools 
you  will  constuct  a  triangle 
when  given  either  its: 
Side,  Side,  Side  (SSS) 
Angle,  Side,  Angle  (ASA)  or 
Side,  Angle,Side  (SAS). 


Figure  2.  Example  of  Text  Version  of  Triangle  Construction  Lesson 


Achievement  and  Attitude  Findings 

As  predicted,  students  who  studied  the  animated  lesson  scored  significantly  higher  than  those  using  the 
graphics  lesson  who  in  turn  scored  significantly  higher  than  those  using  the  text  only  version.  Table  1 
presents  the  ANOVA  summary  table  and  means  scores. 


Table  1 .  ANOVA  Summary  Table  and  Mean  Achievement  Scores  of  Treatment  Groups 


Source  d.f. 

Sum  of  Squares 

Mean  Squares         F-  Ratio 

Probability 

Between  Groups  2 

3890.73 

1945.36  32.34 

<  0.00 

Within  Groups  170 

10226.66 

60.16 

Total  172 

14117.39 

Treatment  Mean 

Text  6.9 

Graphics  1 1 .4 

Animation  18.4 

Total   12.15 


Standard  Deviation  Cases 
5.8  59 
8.1  57 
9.1  57 
9.1  173 


When  the  criterion  was  attitude  toward  mathematics  instruction  by  CBI,  the  animation  group  had 
significantly  higher  attitude  scores  than  the  text  group  and  equivalent  scores  to  the  graphics  group  (Table  2). 
Attitude  was  defined  by  a  Likert-type  scale  designed  to  measure  attitude  toward  the  learning  of  mathematics 
by  CBI. 
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Table  2.  ANOVA  Summary  Table  and  Mean  Attitude  Scores  of  Treatment  Groups 


Source 
Between  Groups 
Within  Groups 
Total 


d.f.  Sum  of  Squares 

1  550.12 

171  13567.27 

172  14117.39 


Mean  Squares 
550.12 
79.34 


F-  Ratio 
6.93 


Probability 

<  0.01 


Treatment 

Text 
Graphics 
Animation 
Total 


Mean 
72.5 
79.4 
79.8 
77.2 


Standard  Deviation 
13.1 
11.5 
11.0 
12.3 


Cases 
59 
57 
57 
173 


Recommendations  for  the  Use  of  Graphics  and  Animation 

Several  recommendations  serve  as  useful  guidelines  for  the  use  of  graphics  and  animation  in  CBI.  These 
recommendations  stem  from  the  research  literature  and  study  cited  above,  supplemented  by  the  personal 
experience  of  the  authors. 

1 .  Analyze  the  relevance  of  graphics/animation  cues  to  the  learning  outcome  and  use  those  cues 
appropriately  in  the  instructional,  practice,  and  testing  situations  relative  to  the  particular  learning 
objectives. 

2.  Opt  for  graphics  in  animations  which  are  simpler  in  design  to  optimize  learning. 

3 .  Examine  graphics/animations  for  these  criteria 

a.  sense  of  perspective,  e.g.,  relative  size,  speed,  and  path  of  motion 

b.  ability  to  convey  the  time-  or  motion-based  aspects  of  animation  in  a  single  viewing, 
alternatively  provide  the  learner  with  multiple  opportunities  to  replay    the  animation. 

c.  clarity  of  representation,  which  may  be  effectively  enhanced  by  the  use  of  text  labels 

d.  the  desirability  of  showing  the  animation  from  multiple  perspectives. 

e.  the  ability  of  the  learner  to  interact  with  and  modify  the  graphic/animation. 

4.  Seek  the  advice/development  expertise  of  a  graphics/animation  specialist. 

5.  Test  out  prototype  lessons  using  different  graphics/anmations  with  your  target  population  of 
learners. 

6.  Test  prototype  lessons  on  the  complete  range  of  target  delivery  machines.  Various  machines  are 
capable  of  different  speeds  of  running  the  animation  or  drawing  the  graphic  and  give  rise  to 
different  effects  which  can  be  quite  different  from  that  intended. 

7.  Complex  animations  may  not  be  optimal  for  beginning  learners  (Rieber,  1990). 

8.  The  real  contribution  of  animation  may  be  in  the  realm  of  interactive  graphics  (Siliasukas,  1986). 
However,  few  have  been  constructed  for  general  education  due  to  the  enormous  complexity  and 
expense  involved. 

9.  Enhance  the  encoding  power  of  graphics  or  animations  by  engaging  the  learners  in  the  creation  and 
use  of  mental  imagery  during  instruction. 

1 0.  Enhance  the  decoding  power  of  graphics  or  animations  by  using  the  same  graphics  and  animations 
in  testing  situations  as  were  used  in  the  instruction 
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CONCLUSIONS 


The  substantial  amount  of  research  conducted  to  determine  effects  of  graphics  on  learning  suggests  a 
high  level  of  effectiveness  for  graphics.  Beyond  thes  generalization  there  are  specific  situations  in  which 
use  of  graphics  must  be  carefully  planned.  Comparatively  much  less  research  has  been  conducted  with 
regard  to  animation  as  an  instructional  tool.  Based  on  results  to  date,  one  might  conclude  that  animation 
has  a  positive  and  significant  effect  upon  learning  outcomes. 

Whatever  media  are  used,  it  is  recommended  that  one  carefully  analyze  the  relevant,  irrelevant,  and 
counterproductive  cues  of  any  learning  task  and  create  or  select  media  which  strike  a  proper  balance  relative 
to  those  cues. 

Consider  the  nature  of  the  target  population  of  learners  relative  to  the  media  chosen  in  terms  of  ability, 
experience,  or  maturity  level. 

Recognize  that  media  has  the  potential  to  add  significant  visual  appeal  to  instruction  but  that  visual 
appeal  is  not  known  to  be  correlated  to  instructional  effectiveness.  At  the  same  time  significant  visual 
appeals  sells  products. 

Consider  enhancing  learning  by  combining  media  with  non-media  instructional  methods  to  optimize 
learning.  For  example,  the  use  of  icons  to  convey  complex  symbols  is  often  subject  to  misinterpretation, 
which  can  be  easily  eliminated  by  addition  of  simple  text.  The  use  of  graphics  and  animation  may  be 
enhanced  by  incorporating  procedures  and  training  in  the  use  of  visualization  techniques. 
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Abstract 


Readily  accessible  computer  technologies  including  on-line  notepads  provide  new 
environments  for  notetaking  research.  Findings  from  this  study  indicate  that  taking  notes 
from  CBI  using  an  on-line  notepad:  promotes  greater  achievement  than  pencil  and  paper 
methods;  is  preferred  by  learners  who  report  higher  confidence  ratings  towards  new 
technologies;  and  promotes  minimal  recording  of  the  learner's  own  thoughts. 

Prior  to  the  paradigmatic  shift  towards  cognitivism  in  the  mid  1970's,  the  significance  of 
notetaking  was  openly  challenged.  Gagne'  states  in  the  1970  addition  of  his  text  The  Conditions  of 
Learning  that  "Most  (students)  may  be  taking  notes,  which  as  far  as  anyone  knows  is  an  entirely  useless 
activity  quite  unrelated  to  learning."  Gage  and  Berliner  in  their  1975  publication  of  Educational  Psychology 
suggest  that  "Notetaking  may  be  effective  only  when  a  student's  short  term  memory  is  relatively  high." 
Despite  of  comments  like  this,  learners  have  been  encouraged  for  decades  to  take  notes  during  lectures  and 
from  readings  and  to  use  them  as  a  source  of  review  (Hartley  and  Davies,  1978). 

The  movement  towards  cognitivism  was  significant  because  it  shifted  the  focus  to  the  learner's 
cognitive  state  and  processing  strategies  as  the  primary  determiners  of  learning.  Wittrock's  generative 
model  of  learning  sought  to  summarize  the  constructive  processes  between  the  individual  and  stimuli 
required  for  learning.  According  to  Wittrock's  model,  when  a  motivated  learner  is  presented  with  a  stimulus 
(lecture  material,  text  readings,  illustrations,  etc.),  the  learner  constructs  and  assigns  meaning  to  it  based  on 
prior  learning.  Previously  learned  information  used  in  constructing  meaning  for  the  stimulus  may  consist 
of  rules,  algorithms,  schemata,  or  image  or  verbal  memories  of  specific  experiences.  The  goal  of  this 
construction  process  is  to  form  associations.  The  more  associations  that  the  learner  can  establish  between 
the  stimulus  and  previously  learned  material,  the  more  meaning  the  stimulus  has  associated  with  it,  thus 
providing  a  better  chance  for  it  to  be  encoded  into  memory  (Wittrock,  1974,  1978). 

It  is  at  this  point,  when  the  learner  is  actively  engaged  in  building  associations  between  what  is 
known  and  that  which  is  to  be  interpreted,  that  the  inherent  value  of  notetaking  as  a  generative  learning 
activity  becomes  apparent.  Generative  activities  such  as  cognitive  mapping,  generating  questions  or 
mnemonic  memory  aids,  underlining,  paraphrasing,  summarizing,  and  notetaking  require  the  learner  to 
knowingly  and  intentionally  relate  new  information  to  their  knowledge  rather  than  passing  over  or 
responding  to  material  without  personal  intervention. 

The  merits  of  notetaking  can  be  viewed  from  two  different  perspectives:  as  a  facilitator  for 
encoding  information  into  memory  or  as  an  external  storage  medium  to  enhance  review  of  the  subject 
matter  (Di  Vesta  &  Gray,  1972).  The  encoding  hypothesis  implies  that  notetaking  is  an  individual  task 
through  which  learners  can  actively  integrate  and  elaborate  upon  what  they  hear,  see,  or  read  with  their  prior 
knowledge.  Once  generated,  the  external  storage  hypothesis  posits  that  the  notes  themselves  function  as  a 
review  tool,  a  form  of  external  storage  device. 

The  majority  of  research  studies  on  not?taking  to  date  have  utilized  live-lecture  situations  and 
conclude  that  notetaking  facilitates  learning  either  through  encoding  or  external  storage  or  a  combination  of 
both  (Hartley,  1983,  Ladas,  1980,  Henk  &  Stahl,  1985).  Only  a  limited  number  of  studies  involving 
notetaking  from  text,  films,  and  television  exist  (Harfiey,  1983).  Research  conducted  utilizing  text 
suggests  that  the  encoding  function  of  notetaking  is  most  significant  (Barnett,  Di  Vesta,  &  Rogozinski, 
1981,  Bretzing  &  Kulhavy,  1979,  Rickards  &  Friedman,  1978)  The  limited  number  of  studies  done  using 
films  and  television  offered  no  insight  as  to  the  function  of  notetaking.  No  research  was  located  which 
utilized  the  computer  as  either  a  medium  used  to  record  notes  or  as  a  source  from  which  notes  were 
generated. 

Looking  forward,  past  the  pencil  and  paper  used  in  traditional  notetaking  research,  are  readily 
accessible  computer  technologies  including  on-line  notepads,  window  environments,  and  notebook  sized 
computers  which  provide  new  environments  for  notetaking  research.  The  computer  technology  which 
facilitates  notetaking  research  in  and  of  itself  is  of  little  interest;  rather  it  is  the  cognitive  effects  of  these 
technologies  on  the  intellectual  performance  of  the  learner  that  is  significant  (Salomon,  Perkins,  & 
Globerson,  1990). 

This  study  sought  to  describe  the  effects  of  technology  on  the  notetaking  strategies  and  behaviors 
of  university  students.  More  specifically,  the  following  questions  were  addressed  in  this  study:  ( 1 )  Given 
a  choice,  do  students  prefer  taking  notes  from  a  computer  tutorial  using  pencil  and  paper  or  the  computer 
itself?  (2)  Is  there  a  relationship  between  the  subject's  confidence  level  towards  using  computer  related 
technology  and  the  method  of  notetaking  selected?  (3)  Does  the  method  of  notetaking  affect  time  on  task 
ox  achievement?  (4)  Does  the  content  of  the  notes  (word  count,  percent  of  information  copied,  and  percent 
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of  main  ideas)  differ  significantly  between  students  who  used  pencil  and  paper  to  take  notes  and  those  that 
used  a  computer? 


Method 

Subjects 

A  student  profile  questionnaire  was  administered  at  a  university  in  southern  Minnesota  to  104 
undergraduates  whose  major  or  minor  was  computer  science.  Computer  science  majors  and  minors  were 
selected  for  this  study  because  their  discipline  required  significant  time  utilizing  computer  technology,  thus 
assuring  adequate  experience  with  computers  prior  to  the  study.  Only  students  who  indicated  they  had 
workedextensively  with  microcomputers  and  word  processing  applications,  and  had  no  prior  knowledge  of 
the  technical  aspects  of  IBM's  AS/400  minicomputer  system  were  eligible  for  selection.  Based  on  these 
criteria,  76  subjects  participated  in  this  study.  Of  this  group,  61  had  a  class  rank  of  senior  and  15  had  a 
rank  of  junior. 
Materials 

Tutorial  Over  a  period  of  five  weeks,  participants  used  either  pencil/paper  or  a  computer  to 
generate  notes  while  reviewing  selections  from  IBM's  AS/400  minicomputer  on-line  tutorial.  The  tutorial 
modules  used  in  this  study  were:  Introduction  to  the  Operating  System;  Operating  System  and 
Architecture  Support;  Equipment  Overview;  Control  Language  Structure;  Attachment  of  Personal 
Computers  and  Other  Devices.  No  time  limit  was  placed  on  viewing  the  modules. 

On-line  notetakin  t>  pad.  The  tutorial  modules  were  accessible  over  a  network  configuration  which 
allowed  the  participants  to  simultaneously  view  a  module  and  access  the  on-line  notetaking  pad  with  a 
single  keystroke  The  computer  notetakinn  pad  consisted  of  a  scrolling  window  that  had  the  same 
formatting,  editing,  printing,  and  copy/paste  capabilities  as  a  word  processor.  The  notepad  was  also 
dynamic;  it  could  be  sized  and  repositioned  on  the  screen  at  any  time. 

Submission  of  notes  Each  week,  after  the  notes  were  submitted,  the  method  used  by  each  subject 
to  take  notes  was  recorded.  In  addition,  the  notes  from  randomly  selected  subjects  from  both  groups  were 
evaluated  weekly  for  word  count,  percentage  of  copied  information,  and  percentage  of  main  ideas. 

Post  test  The  achievement  posttest  consisted  of  fifteen  question:  seven  multiple  and  seven  till  in 
the  blanks  The  questions  included  recall,  appli<  ation  and  synthesis  questions.  Subjects  were  not  allowed 
to  review  their  notes  prior  to  the  test.  The  KR-20  reliability  for  this  test  was  .71. 

Exit  questionnaire.  This  instrument  consisted  of  open  ended  questions  which  sought  to  determine 
why  a  participant  selected  their  dominate  method  of  notetaking.  Additionally,  it  served  as  a  self  report 
method  to  elicit  the  amount  of  time  spent  viewing  and  taking  notes  per  module. 

Procedures  ,  ,,    „,.„„, „j„ 

Subjects  were  selected  for  the  study  based  on  their  responses  to  key  questions  on  the  prestudy 
student  profile  questionnaire.  Prior  to  the  study,  all  subjects  selected  for  the  study  participated  in  a  two 
hour  training  session.  The  study  required  each  participant  to  review  and  take  notes  using  either  pencil  and 
paper  or  the  on-line  notepad  from  one  tutorial  module  each  week  for  five  weeks.  All  notes  were  monitored 
weekly  tor  the  notetaking  method  used.  In  addition,  randomly  selected  notes  from  both  the  pencil  and  paper 
and  on-line  notepad  group  were  evaluated  weekly  for  word  count,  percentage  of  copied  information  and 
percentage  of  main  ideas.  After  the  submission  of  the  last  weeks'  notes,  each  student  completed  an 
achievement  posttest  and  completed  an  exit  questionnaire.  Subjects  completed  all  work  on  their  own  time 
and  were  given  extra  points  equivalent  to  ten  percent  of  their  grade  in  one  computer  science  class  during  the 
quarter  ot^tu  y.^  ^       ^       ^  ^  ^  ^  desjgned  to:  (1 }  review  the  criteria 

and  benefits  of  good  notes  regardless  of  the  medium;  (2)  define  the  objectives  for  each  module  used  ,n  the 
turly  (3  provide  hands-on  experience  accessing  the  AS/400  tutorial  modules  and  on-line  notepad.  Durmg 
the  session  participants  were  also  instructed  that  at  any  time  during  the  study  they  could  select  the  alternate 

n0tetokinf,5o»  'of  Subjects  for  Weekly  Monitoring.  In  addition  to  each  participant's  method  of 
notetaking  being  recorded  weekly,  notcs'taken  by  subjects  from  both  groups  were  randomly  selected  tor 
week evaluation.  After  the  first  week's  notes  were  submitted,  four  subjects  (23.5  percent)  who  used 
nenci  and  paper  and  1 2  (22  percent)  who  used  the  on-line  notepad  for  notetaking  were  randomly  .elected  for 
wee > monitor  n during  th'e  entire  study.  Interestingly  enough,  each  monitored  participant  used  he ^ame 
Sod  Selected  for  notetaking  the  first  week  throughout  the  remainder  of  the  study.  Also  an  ^.onal 
five  subjects  from  the  pencil  and  paper  group  and  12  from  the  on-line  notepad  group  were  randomly  selected 


each  week  for  weekly  monitoring.  Thus,  the  notes  from  approximately  44  percent  of  the  study's 
participants  were  evaluated  weekly  for  word  count,  percent  of  copied  material,  and  percent  of  main  ideas. 

Reliability  of  Random  Evaluation  Scoring  .  Prior  to  the  beginning  of  the  study,  this  author  and  a 
graduate  research  assistant  in  computer  science  reviewed  each  of  the  topics  in  the  tutorial  used  in  the  study 
and  each  compiled  an  set  of 'ideal'  notes  about  the  topics.  Agreement  was  easily  reached  regarding  the  main 
ideas  presented  in  each  tutorial  module.  A  printout  which  contained  the  text,  diagrams,  and  questions  found 
in  the  tutorial  was  made  available  to  each  reviewer. 

Scoring  reliability  for  the  participants'  notes'  word  count,  percentage  of  main  ideas,  and  percentage 
of  information  copied,  was  estimated  by  having  both  reviewers  score  a  sample  of  notes  taken  on  the  same 
topic  from  the  pencil  and  paper  and  on-line  notepad  groups.  Using  this  method,  we  were  able  to  ensure  that 
the  notes  selected  for  review  on  any  given  topic  were  scored  by  two  people.  For  the  sets  of  notes  used  in 
the  reliability  check,  disagreements  were  resolved  by  rechecking  the  notes  and  discussion.  Based  on  the 
review  of  75  sets  of  notes,  the  average  percent  of  reviewer  agreement  was:  95.4  percent  for  word  count.  98.3 
percent  for  percent  main  ideas,  and  96  percent  for  percent  of  information  copied. 

Dominant  Notetaking  Method.    At  the  conclusion  of  the  study,  subjects  were  assigned,  for 
statistical  calculations,  to  one  of  three  groups  (pencil/paper,  on-line,  or  combination  of  bothWhods)  based 
on  the  dominate  method  used  to  record  their  weekly  notes.  To  be  placed  in  either  the  pencil/paper  or  on-line 
group,  students  must  have  selected  that  method  for  four  or  more  weeks.  All  but  three  students  participating 
in  the  study  were  originally  assigned  to  one  of  these  two  groups.  In  other  words,  it  appeared  that  once  a 
subject  selected  a  method  to  record  notes,  that  method  was  used  consistently  throughout  the  study.  Be  ause 
of  their  limited  number,  the  three  subjects  who  comprised  the  third  group  were  not  used  in  the  statistical 
calculations. 

Posttest  and  exit  questionnaire.  Upon  reviewing  the  five  tutorial  modules,  subjects  completed  an 
achievement  test  and  the  exit  questionnaire. 

Data  Analysis.  Statistical  calculations  were  carried  out  using  SPSSX  software.  Significance  level 
was  set  at  p  =  .05.  "  c 

Results 

Achievement 

Means  and  standard  deviations  for  the  posttest  scores  are  given  in  Table  I .  The  on-line  notetakers  (M  = 
7.04)  performed  better  on  the  post  test  than  those  using  pencil  and  paper  (M  =  5.29).  One  way  ANOVA 
results  revealed  that  the  difference  between  the  post  test  means  were  significant  [F(  1 ,71 )  =  4.5 19,  p  =  .037], 
Two-way  ANOVA  results  yielded  no  significant  main  effect  for  major  or  minor  |F(  1 ,69)  =  1 .934,  p  =  .1691 
or  the  type  of  notetaking  method  |F(1.69)  =  2.244,  p  =  .139].  The  interaction  effect  was  not  significant 
[F(1.69)  =  .463,/7  =  .498], 

Table  1 :  Means  and  Standard  Deviations  for  Achievement  

Group  N  Mean  SD   ~ 

Pencil/paper  ]7  5.99  2.17 

On-line  notepad  56  7.04  3.15 


Subjects'  Use  of  Notetaking  Methods 

The  number  of  subjects  and  the  percentage  of  computer  science  majors  and  minors  who  used  pencil 
and  paper  or  on-line  notepad  as  their  dominate  notetaking  method  is  shown  in  Table  2.  As  previously 
described,  subjects  were  assigned  to  a  notetaking  group  based  on  their  weekly  notetaking  method. 


Table  2.  Composition  of  Notetaking  Groups 


Group 

N 

Percent 
of  Total 

Percent  of 
Group  Total 

Percent  of" 
Major/Minors 

Pencil/paper 

17 

23.29 

Computer  science  majors 

4 

23.53 

9.3 

Computer  science  minors 

13 

76.47 

■13.3 

On-line  notepad 

56 

76.71 

Computer  science  majors 

39 

64.29 

90.7 

Computer  science  minors 

17 

30.36 

56.7 
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Attitudes 

Several  Likert-type  items  on  the  prestudy  student  profile  questionnaire  were  used  to  assess  the 
subjects'  confidence  working  with  computers,  word  processors,  and  other  new  technologies.  The  means  and 
standard  deviations  for  these  questions  are  presented  in  Table  3.  The  ranking  scale  consisted  of  the  value 
five  associated  with  low  confidence  rating  and  the  value  one  associated  with  a  high  confidence  rating.  The 
on-line  notetakers  responded  consistently  more  positively  towards  using  computer?  and  technologies  than 
the  pencil  and  paper  group. 

Table  3:  Means  and  Standard  Deviations  for  Attitude  Scored  Associated  with  Using  Computers  and  Other 
Technologies  


Groups 


Pencil/Paper  CN= 

171 

On-Line  Notepad  (N=56) 

Ouestions 

Mean 

SD 

Mean 

SD 

Use  of  microcomputers 

1.65 

.71 

1.29 

.60 

Use  of  word  processors 

1.88 

.86 

1.36 

.65 

Use  of  computer  related  technologies 

2.53 

1.01 

1.48 

.69 

Acceptance  of  new  methods 

to  perform  tasks 

2.59 

1.06 

1.76 

.77 

Adjust  to  using  new  technologies 

2.47 

.87 

1.60 

.78 

One-way  ANOVA  results  from  follow-up  comparisons  between  the  mean  scores  revealed  that  the  differences 
between  the  means  for  each  question  were  significant:  (use  of  microcomputers  [F(  1,71)  =  4.23,  p  =  .04]; 
use  of  word  processors  [F(l,71)  =  7.09,  p  =  .01];  use  of  computer  related  technologies  [F(l,71)  =  23.48,/? 
<.  01];  acceptance  of  new  methods  to  perform  tasks  [F(  1,71)  =  12.35,  p  <  .01;  adjust  to  using  new 
technologies  [F(  1,71  )  =  15.16,  p  <.01]). 

In  the  exit  questionnaire,  subjects  were  asked  to  rank  possible  reasons  for  selecting  their  dominate 
notetaking  method  used  during  the  study.  A  ranking  of  one  indicated  that  it  was  the  most  important  reason 
for  selecting  the  method  while  the  rank  of  six  signified  it  was  the  least  important.  Reasons  that  were 
marked  as  not  important  in  the  decision  making  process      t  not  included  in  the  calculations.  The  means 
and  standard  deviations  for  each  possibility  are  given  in  Taole  4.  Students  selected  the  on-line  notepad 
because  of  the  speed  and  the  ease  of  reading  the  notes  once  recorded.  It  appeared  the  other  group  viewed  the 
pencil  and  paper  as  the  easiest  way  to  record  their  thoughts. 

Table  4:  Means  and  Standard  Deviations  for  Reason  for  Selecting  a  Notetaking  Strategy__ 

 Groups  


Pencil/Paper  (N=17)  On-Line  Notepad  (N=56) 


Reasons 

Mean 

SD 

Mean 

SD 

Faster  method  to  record  notes 

3.77 

.62 

1.62 

"0 

Notes  easier  to  read,  study  froma 

4.86 

.57 

2.96 

.26 

Easiest  way  to  record  thoughts 

3.22 

.48 

4.60 

.30 

Better  environment  to  learn  from 

4.82 

.50 

4.40 

.28 

Used  to  working  in  a  window 

environment 

4.50 

.53 

4.79 

.31 

Not  worked  with  an  electronic 

notepad  before 

4.91 

.56 

5.51 

.36 

Not  worked  on  the  AS/400 

computer  system  before 

5.91 

.69 

5.94 

.34 

4 1)  J 

O  455 

ERIC 


Time  on  Task 

Subjects  were  asked  on  the  exit  questionnaire  to  estimate  the  time  spent  reviewing  and  recording 
notes  for  ea'-h  of  the  modules.  The  means  and  standard  deviations  for  the  average  time  spent  per  module  is 
reported  in  Table  5.  A  follow-up  one-way  ANOVA  indicated  no  significance  between  the  means  of  the  two 
groups  [F(l,71)  =  .03,p  =  ..87]. 


Table  5:  Means  and  Standard  Deviations  for  Time  on  Task 


GrouD 

N 

Mean 

SD 

Pencil/paper 

17 

39.99 

15.91 

On-line  notepad 

56 

39.32 

13.82 

Word  Count,  Percent  Information  Copied  and  Percent  Main  Ideas 

The  monitored  subjects  using  the  on-line  notepad  had  significantly  higher  word  counts,  percentage 
of  copied  material,  and  percentage  of  main  ideas  than  those  using  pencil  and  paper.  The  means  and  standard 
deviations  for  word  count,  percent  information  copied,  and  percent  of  main  ideas  for  the  groups  monitored 
weekly  are  given  in  Table  6.  More  information  was  copied  directly  from  the  tutorial  by  the  group  using 
on-line  notepad  (M  =  90. 12  percent)  than  by  the  pencil/paper  group  (M  =  46. 17  percent).  This  may  have 
accounted  for  the  higher  word  count  associated  with  the  on-line  notepad  group.  The  notes  from  the  on-line 
group  also  contained  more  main  ideas  (M  =  86.63  percent)  than  the  pencil/paper  group  (M  =  72.75  percent). 

Table  6.  Means  and  Standard  Deviations  for  Parameters  Monitored  Weekly   

Parameter  

Word  Count 

Percent  Copied 

Percent  Main  Ideas 


Grouping 

N 

Mean 

SD 

Pencil/Paper 

13 

175.67 

132.94 

On-line  notepad 

43 

34'  60 

179.72 

Pencil/Paper 

13 

46.17 

24.37 

On-line  notepad 

43 

90.12 

15.89 

Pencil/Paper 

13 

72.75 

31.07 

On-line  notepad 

43 

86.43 

16.19 

One-way  ANOVAS  revealed  differences  between  the  pencil  and  paper  and  on-line  notepad  group  on 
the  following  parameters:  average  word  count  [F  (1,54)  =  8.799,  p  =  .005,  percentage  of  notes  copied  [F 
(1,54)  =  55.517,  p  <. 000],  percent  main  ideas  [F  (1,54)  =  4.25,  p  =.04]. 


Discussion 

This  study  examined  the  following:  if  one  method  of  notetaking  promoted  a  higher  level  of 
achievement;  if  students  favored  taking  notes  from  a  computer  tutorial  with  a  microcomputer  or  pencil  and 
paper;  and  if  the  content  and  structure  of  the  notes  differed  between  platforms  used  to  record  notes. 

Results  of  this  study  show  that  taking  notes  using  the  on-line  notepad  is  the  most  effective 
method  of  notetaking,  as  assessed  by  the  post  test,  regardless  of  whether  the  participant's  major  or  minor  is 
computer  science.  It  appears  that  the  ability  to  copy  and  paste  text  as  well  as  generating  notes  provides  an 
environment  which  promotes  more  achievement  than  traditional  pencil  and  pap.  r  methods. 

Results  of  this  study  showed  that  subjects  preferred  using  the  on-line  notepad  rather  than  traditional 
pencil  and  paper.  An  overwhelming  percentage  of  computer  science  majors  and  approximately  half  of  the 
minors  consistently  used  the  on-line  method  to  record  notes.  Participants  who  selected  the  on-line  method 
reported  higher  confidence  ratings  towards  using  microcomputer,  word  processors  and  new  technologies  than 
those  using  pencil  and  paper.  This  discounts  any  suggestion  that  a  novelty  effect  may  have  been  associated 
with  the  use  of  uV  on-line  notepad.  Instead,  these  finding  indicate  that  the  more  confident  the  subjects 
were  with  using  the  technology,  the  more  it  was  utilized. 
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The  results  of  the  present  study  also  revealed  that  notes  taken  by  the  on-line  group  consisted 
mainly  of  text  that  was  copied  from  the  tutorial  and  pasted  to  the  notepad  while  the  notes  generated  iy  the 
pencil/paper  group  reflected  more  of  their  own  thoughts.  The  copy  and  paste  strategy  employed  by  the  on- 
line group  may  have  impacted  the  group's  word  count  and  percent  of  main  ideas  tabulations  as  well. 

As  previously  enumerated,  an  important  criterion  used  by  both  groups  to  select  a  notetaking 
method  was  how  quickly  notes  could  be  recorded  utilizing  the  method.  When  examined,  no  significant 
difference  was  found  between  the  two  groups'  average  time  per  module  spent  reviewing  and  taking  notes. 

As  computer  related  technologies  evolve,  current  images  are  redefined;  books  have  become  disks 
and  pencils  have  been  transformed  into  keyboards  and  mice.  Although  it  is  tempting,  our  primary  attention 
should  remain  focused  on  the  programs  and  .ools  which  can  be  used  with  the  technology  and  the  activities 
they  afford. 

In  order  for  the  computer  to  function  as  a  tool  to  enhance  cognition,  an  individual  must  seek  to 
develop  an  intellectual  partnership  with  technology.  As  demonstrated  in  this  study,  the  strength  of  the 
partnership  relies  heavily  on  how  confident  the  individual  is  in  using  the  technology,  and  whether  the 
activities  promoted  by  the  technology  encourage  active,  mindful  engagement  on  the  part  of  the  learner. 

As  technology  advances,  its  benefits  may  not  be  realized  automatically.  Further  research  should 
focus  on  the  development  and  evaluation  of  activities,  situations,  and  computer  applications  specifically 
crafted  to  promote  engaging  partner  ..ps  between  the  learnt r  and  the  technology.  Research  should  also 
focus  on  how  cognitive  effects,  such  as  the  development  of  problem  solving  strategies  arrived  at  through 
such  partnerships,  ca^  become  more  transferable  to  situations  which  do  not  involve  technology. 

Specific  to  this  study,  there  are  five  areas  which  merit  further  investigation.  First,  identify  the 
cognitive  strategies  employed  by  students  who  prefer  using  an  on-line  notepad  for  notetaking.  Second, 
assess  whether  the  on-line  method  of  notetaking  is  as  beneficial  when  students  are  not  allowed  to  use  the 
copy/paste  function  but  rather  must  generate  original  notes.  Third,  examine  whether  the  learner's 
familiarity  with  the  content  of  the  tutorial  impacts  the  method  of  notetaking  selected  or  the  amount  of 
material  copied  directly  from  the  tutorial  and  placed  in  the  notes.  Fourth,  identify  the  preferred  notetaking 
method  of  learners  form  other  discipline  areas  and  grade  levels.  Fifth,  examine  the  relationship  between 
using  technology  to  generate  notes  over  an  extended  period  of  time  and  the  process  and  product  functions  of 
notetaking. 
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My  First  Flame 

In  April  1994, 1  was  "flamed"  on  the  Internet,  a  '90s  phenomenon  that  has  been  portrayed  in  publications  as 
diverse  as  The  New  Yorker  (Seabropk,  1994)  and  The  Cfirbnicle  of  Higher  Education  (Lernisch,  1995). 
Although  what  exactly  constitates^a'"flame''  'in  the"  rapi8fy  WoTvmg  '"metaverse"  (Stephenson,  1993)  is  a 
matter  of  much  debate,  I  can  vividly  recall  the  feelings  of  shock  and  anger  that  swept  through  me  when  I 
read  the  note  calling  me  a  "jerk"  on  a  "listserv"  shared  by  hundreds  of  members  around  the  world. 

It  all  began  last  spring  when  I  read  two  queries  from  doctoral  students  on  the  Qualitative  Research  for  the 
Human  Sciences  listserv  <QUALRS-L@uga.cc.uga.edu>.  Both  students  came  from  large  public 
institutions  of  higher  education,  one  in  the  USA  and  the  other  in  Canada.  The  first  student  wrote  that  she 
intended  to  focus  her  dissertation  research  on  the  quality  of  "discourse"  that  takes  place  in  cafes  and  coffee 
shops  located  inside  bookstores.  She  complained  that  she  had  found  no  "literature"  on  this  topic  and  asked 
the  listserv  participants  for  some  guidance.  The  second  student  announced  that  he  was  preparing  a 
dissertation  prospectus  centered  on  the  question  of  how  people  learned  about  opportunities  to  take  SCUBA 
diving  lessons  and  what  motivated  them  to  register  for  such  courses.  He  also  sought  directions  to  relevant 
literature  and  advice  from  the  listserv  membership. 

After  pondering  these  queries,  I  posted  a  message  asking  whether  faculty  members  at  taxpayer-supported 
universities  have  a  moral  responsibility  to  guide  their  students  toward  "socially  responsible"  research 
questions.  In  my  posting,  I  suggested  that  in  the  face  of  problems  such  as  adult  illiteracy,  attacks  on  public 
education,  "at-risk"  students,  homelessness,  AIDS,  and  the  like,  faculty  members  should  attempt  to  inspire 
in  students  a  dedication  to  research  that  would  "make  a  difference." 

Soon  after  posting  my  note,  the  graduate  student  who  had  sought  help  with  his  SCUBA  query  "flamed"  me 
with  his  "jerk"  note  in  which  he  went  on  to  criticize  my  "attack"  on  his  freedom  to  address  whatever 
research  questions  interested  him,  especially  given  he  was  a  taxpayer  as  well.  A  small  grass  fire  of  flames 
then  erupted  as  several  listserv  members  castigated  the  student  for  calling  me  a  jerk,  some  agreed  with  my 
critique,  and  others  defended  the  perspective  that  the  social  relevance  of  doctoral  dissertation  research  (or  any 
educational  research)  was  irrelevant.  No  resolution  of  this  issue  was  reached  on  the  listserv,  but  I  was 
especially  impressed  by  the  response  of  an  education  professor  from  a  large  land  grant  university  in  the 
USA  who  agreed  with  my  criticism,  but  went  on  to  suggest  that  much  of  the  research  he  has  read  in  the 
field  of  instructional  technology  could  be  subjected  to  a  similar  critique.  This  prompted  me  to  ponder  the 
social  relevancy  of  research  in  our  field. 

Is  Instructional  Technology  Research  Socially  Relevant? 

Social  relevance  is  an  issue  that  is  obvious'"  subject  to  much  debate.  One's  age,  race,  gender, 
socioeconomic  status,  education,  religion,  ^uiitical  allegiance,  and  many  other  factors  are  likely  to  influence 
one's  interpretation  of  the  social  relevance  of  any  given  research  study.  Nevertheless,  for  the  sake  of  this 
analysis,  I  will  attempt  to  define  social  relevance  with  respect  to  scientific  inquiry.  My  definition  is  based 
upon  the  following  principles  that  guide  scientific  research  (derived  from  Casti,  1989): 

•  Science  is  an  ideology  that  consists  of  a  cognitive  structure  concerning  the  nature  of  reality  together  with 
processes  of  inquiry,  verification,  and  peer  review. 

•  Views  of  reality  differ  according  to  one's  philosophy  of  science,  e.g.,  realism  maintains  that  an  objective 
reality  exists,  instrumentalism  asserts  that  reality  is  the  readings  noted  on  measuring  instruments,  and 
relativism  claims  that  reality  is  what  the  community  says  it  is. 

•  Scientific  research  is  a  social  activity  that  has  certain  standards  and  norms,  e.g.,  it  should  not 
intentionally  harm  humans  and  it  must  be  able  to  be  replicat^''  by  other  researchers. 

•  Socially  responsible  research  in  education  adheres  to  the  basic  principles  listed  above  while  at  the  same 
time  it  addresses  problems  that  detract  from  the  quality  of  life  for  individuals  and  groups  in  society, 
especially  those  problems  related  to  learning  and  human  development. 

In  the  view  of  some,  instructional  technology  research  might  lay  claim  to  a  blanket  imprimatur  with 
respect  to  being  "socially  responsible."  After  all,  at  some  level,  all  instructional  technology  research  can  be 
said  to  focus  on  questions  of  how  people  learn  and  perform,  especially  with  respect  to  how  learning  and 
performance  are  influenced,  supported,  or  perhaps  even  caused  by  technology.  As  long  as  research  is 
focused  on  learning  and  performance  problems,  and  adheres  to  the  principles  listed  above,  it  would  seem  to 
be  socially  responsible. 
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Others  in  the  research  community  argue  that  concern  for  the  social  responsibility  of  research  in  instructional 
technology  or  any  other  field  is  ludicrous.  They  maintain  that  the  goal  of  research  is  knowledge  in  and  of 
itself,  and  that  whether  research  is  socially  responsible  is  a  question  that  lies  outside  the  bounds  of  science 
(cf.,  Carroll,  1973).  In  my  experience,  researchers  in  the  natural  sciences  such  as  biology  and  chemistry  do 
not  often  concern  themselves  with  the  relevance  question,  but  this  is  a  debate  that  has  raged  for  decades 
among  educational  researchers  (see  Note  1).  For  example,  as  reported  by  Farley  (1982),  Nate  Gage,  a  past 
president  of  the  American  Educational  Research  Association  (AERA),  has  been  a  staunch  defender  of  the 
notion  that  the  goal  of  basic  research  in  education  is  simply  "more  valid  and  more  positive  conclusions"  (p. 
12)  whereas  another  past  president  of  AERA,  Robert  Ebel,  proclaimed: 

....the  value  of  basic  research  in  education  is  severely  limited,  and  here  is  the  reason.  The  process  of 
education  is  not  a  natural  phenomenon  of  the  kind  that  has  sometimes  rewarded  scientific  investigation.  It 
is  not  one  of  the  givens  in  our  universe.  It  is  man-made,  designed  to  serve  our  needs.  It  is  not  governed  by 
any  natural  laws.  It  is  not  in  need  of  research  to  find  out  how  it  works.  It  is  in  need  of  creative  invention 
to  make  it  work  better,  (p.  18,  Ebel's  italics). 

In  my  opinion,  Ebel's  stance  (with  which  I  agree)  is  directly  relevant  to  the  issue  of  socially  responsible 
research  in  instructional  technology.  There  is  little  social  relevance  in  research  studies  that  are  largely 
focused  on  understanding  "how"  instructional  technology  works  without  substantial  concern  for  how  this 
understanding  makes  education  better.  On  the  other  hand,  there  is  considerable  social  relevance  in  IT. 
research  studies  that  are  largely  focused  on  making  education  better  (and  which  in  the  process  may  also  help 
us  understand  more  about  how  instructional  technology  works). 

Most  of  the  research  in  instructional  technology  is  grounded  in  a  "realist"  philosophy  of  science,  i.e., 
conducted  under  the  assumv  tion  that  education  is  part  of  an  objective  reality  governed  by  natural  laws  and 
therefore  can  be  studied  in  a  manner  similar  to  other  natural  sciences  such  as  chemistry  and  biology.  If  this 
assumption  about  the  nature  of  the  phenomena  we  study  is  erroneous  (and  I  believe  it  is),  then  we 
inevitably  ask  the  wrong  questions  in  our  research.  Further,  even  if  there  are  underlying  laws  that  influence 
learning,  the  complexity  inherent  in  these  laws  may  defy  our  ability  to  perceive,  much  less  control,  them 
(Casti,  1994).  As  Cronbach  (1975)  pointed  out  two  decades  ago,  our  empirical  research  may  be  doomed  to 
failure  because  we  simply  cannot  pile  up  generalizations  fast  enough  to  adapt  our  instructional  treatments  to 
the  myriad  of  variables  inherent  in  any  given  instance  of  instruction. 

Of  course,  I  am  not  the  first  person  to  express  concerns  about  the  nature  of  research  in  instructional 
technology.  I  adopted  the  title  of  this  paper  from  one  published  twenty-seven  years  ago  by  Keith  Mielke 
(1968)  titled  "Questioning  the  Questions  of  ETV  Research."  Other  critics  of  the  questions  and  methods  of 
research  in  instructional  technology  include  Lumsdaine  (1963),  Schramm  (1977),  Clark  (1983),  and 
Salomon  (1991).  The  debate  about  the  nature  of  reality  and  the  conduct  of  research  in  our  fteld  continues  as 
evidenced  by  the  recent  spate  of  articles  focused  on  the  question  of  "Does  media  influence  learning?"  (Clark, 
1994a,  b;  Jonassen,  Campbell,  &  Davidson,  1994;  Kozma,  1994a,  b;  Morrison,  1994;  Reiser,  1994;  Ross, 
1994a,  b;  Shrock,  1994;  Tennyson,  1994;  Ullmer,  1994).  However,  few  critics  have  dealt  directly  with 
questions  of  whether  instructional  technology  research  is,  can  be,  or  should  be  socially  responsible.  That  is 
the  purpose  of  this  paper. 

The  State  of  Instructional  Technology  Research 

Before  returning  to  the  issue  of  the  social  relevance  of  instructional  technology  research,  it  is  necessary  to 
examine  the  state  of  research  in  the  field  today.  To  accomplish  this,  I  reviewed  the  contents  of  two  of  the 
primary  research  journals  in  the  field,  the  Educational  Technology  Research  and  Development  (ETR&D) 
journal  and  the  Journal  of  Computer-based  Instruction  (JCBI)  over  the  periods  1 989-94  for  ETR&D  and 
1 988-93  for  JCBI  (see  Note  2). 

For  this  review,  I  originally  intended  to  use  a  research  article  classification  scheme  developed  by  Dick  and 
Dick  (1989)  (see  Figure  1),  but  my  initial  attempts  to  categorize  articles  using  that  scheme  led  to  several 
difficulties,  especially  in  terms  of  classifying  studies  that  were  primarily  interpret!  vist  in  intent  and 
naturalistic  in  method,  e.g.,  Neuman  (1991). 
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Literature  review 

a  summary  of  a  body  of  literature,  sometimes  as  a  critique  and  sometimes  as 
a  statement  of  the  state  of  the  art. 

Methodological  article 

a  new  model  or  procedure  for  carrying  out  a  technical  activity. 

Theoretical  article 

one  which  primarily  draws  upon  and  contributes  to  the  theoretical  literature 
of  the  field. 

Empirical  and 
experimental  studies 

all  studies,  other  than  evaluations,  which  use  data  in  order  to  draw 
conclusions. 

Descriptive  study 

a  presentation  of  information  about  a  particular  program  or  event,  with  little 
or  no  use  of  data. 

Evaluation  study 

a  presentation  of  data  and  information  to  describe  the  effectiveness  of  a 
particular  program  or  method,  usually  in  an  applied  setting. 

Professional  article 

a  description  of  topics  dealing  with  the  profession  of  instructional 
technology,  such  as  determination  of  competencies  or  descriptions  of 
internship  programs. 

Figure  1 .  Research  article  classification  scheme  from  Dick  and  Dick  (1989). 


After  reflection  and  consultations  with  several  research  experts,  I  modified  the  classification  scheme  (see 
Note  3)  This  new  classification  scheme  represents  an  effort  to  distinguish  between  the  goals  of  research  and 
the  methods  of  research.  First,  I  propose  that  most  research  studies  in  instructional  technology  can  be 
classified  according  to  the  six  research  goals  represented  in  Figure  2.  This  scheme  is  partially  based  upon 
discussions  of  research  "paradigms"  that  have  dominated  educational  research  literature  in  recent  years.  For 
example,  according  to  Soltis  (1992),  there  are  currently  "three  major  paradigms,  or  three  different  ways  of 
investigating  important  aspects  of  education"  (p.  620)  used  in  educational  research:  1)  the  positivist  or 
quantitative  paradigm,  2)  the  interpretivist  or  qualitative  paradigm,  and  3)  the  critical  theory  or  neomarxist 
paradigm.  Although  the  "paradigm  debate"  literature  is  fascinating,  I  do  not  feel  that  the  three  categories 
presented  by  Soltis  (1992)  and  others  (e.g.,  Schubert  &  Schubert,  1990)  capture  the  full  breadth  of  research 
goals  in  the  field  of  instructional  technology.  At  the  same  time,  there  may  be  other  goals  of  inquiry  that 
are  not  be  included  in  this  scheme. 


Theoretical 

research  focused  on  explaining  phenomena  through  the  logical  analysis  and 
synthesis  of  theories,  principles,  and  the  results  of  other  forms  of  research 
such  as  empirical  studies. 

Empirical 

research  focused  on  determining  how  education  works  by  testing  conclusions 
related  to  theories  of  communication,  learning,  performance,  and  technology. 

Interpretivist 

research  focused  on  portraying  how  education  works  by  describing  and 
interpreting  phenomena  related  to  human  communication,  learning, 
performance,  and  the  use  of  technology. 

Postmodern 

research  focused  on  examining  the  assumptions  underlying  applications  of 
technology  in  human  communication,  learning,  and  performance  with  the 
ultimate  goal  of  revealing  hidden  agendas  and  empowering  disenfranchised 
minorities. 

Developmental 

research  focused  on  the  invention  and  improvement  of  creative  approaches  to 
enhancing  human  communication,  learning,  and  performance  through  the 
use  of  technology  and  theory. 

Evaluation 

research  focused  on  a  particular  program,  product,  or  method,  usually  in  an 
applied  setting,  for  the  purpose  of  describing  it,  improving  it,  or  estimating 
its  effectiveness  and  worth.  £\ 

Figure  2.  Research  goal  classification  scheme. 
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Second,  given  the  aforementioned  desire  to  separate  the  goals  of  research  studies  from  the  methodologies 
employed  in  them,  I  propose  the  methodology  classification  scheme  represented  in  Figure  3.  Of  course, 
there  are  numerous  methods  available  to  researchers  in  instructional  technology  (cf.,  Driscoll,  1995),  but  for 
the  sake  of  simplicity,  these  five  methodological  groupings  provide  sufficient  discrimination  to  allow  the 
analysis  represented  below. 


Quantitative 

experimental,  quasi-experimental,  correlational,  and  other  methods  that 
primarily  involve  the  collection  of  quantitative  data  and  its  analysis  using 
inferential  statistics. 

Qualitative 

observation,  case-studies,  diaries,  interviews,  and  other  methods  that 
primarily  involve  the  collection  of  qualitative  data  and  its  analysis  using 
grounded  theory  and  ethnographic  approaches. 

Critical  Theory 

deconstruction  of  "texts"  and  the  technologies  that  deliver  them  through  the 
search  for  binary  oppositions,  hidden  agendas,  and  the  disenfranchisement  of 
minorities. 

Literature  Review 

various  forms  of  research  synthesis  that  primarily  involve  the  analysis  and 
integration  of  other  forms  of  research,  e.g.,  frequency  counts  and  meta- 
analyses. 

Mixed-methods 

research  approaches  that  combine  a  mixture  of  methods,  usually  quantitative 
and  qualitative,  to  triangulate  findings. 

Figure  3.  Research  methods  classification  scheme. 

The  combination  of  the  goal  classification  and  the  methods  classification  schemes  yields  a  matrix  of 
research  goals  by  research  methods.  Figure  4  presents  my  analysis  of  the  research  articles  published  in 
ETR&D  (1989-1994).  There  were  one  hundred  and  four  articles  published  in  the  research  section  of 
ETR&D  in  the  six  years  from  1989  through  1994  (see  Note  4).  Not  every  article  could  be  classified 
according  to  the  classification  matrix  illustrated  in  Figure  4.  Six  "methodological  articles"  (presenting  a 
new  method  or  procedure  for  carrying  out  research)  and  three  "professional  articles"  (analyzing  the  state  of 
the  profession  of  instructional  technology)  are  not  included  in  Figure  4. 
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Figure  4.  Classification  of  ETR&D  articles  (1989-94). 

Figure  5  is  an  analysis  of  the  research  articles  published  in  JCBI  (1988-1993).  There  were  one  hundred  and 
twenty-nine  articles  published  in  JCBI  from  1988  through  1993.  Five  "methodological  articles"  and  one 
"professional  article"  are  not  included  in  Figure  5. 
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Figure  5.  Classification  of  JCBI  articles  (1988-93). 
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There  are  some  obvious  trends  in  the  articles  that  appeared  in  ETR&D  and  JCBI  during  the  respective 
review  periods.  First,  the  most  common  type  of  article  in  either  publication  is  empirical  in  intent  and 
quantitative  in  method.  Thirty-nine  articles  (38%  of  the  total  104)  in  ETR&D  and  fifty-six  articles  (43%  of 
the  total  129)  in  JCBI  fall  into  the  "empirical-quantitative"  cell  of  the  matrix. 

The  next  largest  subset  of  articles  in  these  publications  can  be  classified  as  theoretical  in  intent  and 
employing  literature  review  as  the  primary  method.  I  was  liberal  in  my  classification  of  articles  into  this 
category.  For  example,  I  assigned  all  of  the  aforementioned  media  debate  articles  into  this  classification 
(cf,  Clark,  1994a,  b;  Kozma,  1994a,b).  The  extent  to  which  literature  review  methods  were  actually  used 
in  these  articles  varies  greatly. 

Another  trend  that  stands  out  is  the  paucity  of  interpretivist  articles  (one  in  ETR&D  and  three  in  JCBI) 
during  this  time.  This  seems  surprising  given  the  numerous  applications  of  the  "Constructivist- 
Hermeneutic-Interpretivist-Qualitative  Paradigm"  in  other  fields  of  education  (cf.,  Eisner,  199 1).  Although 
Neuman  (1989),  Driscoll  ( 1995),  Robinson  (1995)  and  others  promote  interpretivist  approaches  to  research 
in  instructional  technology,  interpretivist  research  reports  rarely  find  their  way  into  our  publications. 

Developmental  research  studies  are  also  scarce  in  each  of  these  publications.  With  respect  to  ETR&D,  it 
may  be  that  most  developmental  research  studies  appear  in  the  development  section  of  the  journal,  but  this 
is  a  hypothesis  that  has  not  been  investigated.  Other  possible  explanations  are  that  instructional 
technologists  rarely  conduct  developmental  research,  those  that  do  have  too  little  time  to  report  it,  or  the 
review  panels  for  the  journals  do  not  recognize  this  approach  as  legitimate  research. 

The  complete  absence  of  any  articles  in  these  journals  that  are  postmodern  in  intent  or  that  employ  critical 
theory  as  a  methodology  is  disappointing,  but  not  too  surprising.  First,  Hlynka  and  Belland's  (1991) 
volume  on  the  application  of  postmodern  criticism  to  instructional  technology  may  not  be  widely  known. 
Second,  the  gatekeepers  of  ETR&D  and  JCBI  appear  to  have  strong  preferences  for  empirical  research 
employing  quantitative  methods.  They  may  be  unwilling  or  unable  to  entertain  such  radical  departures  from 
standard  research  methods  as  have  been  proposed  by  Yearn  an  (1994)  and  other  critical  theorists. 

An  interesting  difference  between  the  two  journals  is  the  percentage  of  articles  that  are  evaluative  in  intent. 
Only  nine  (9%)  of  the  articles  in  ETR&D  were  evaluation  reports  during  this  period  whereas  thirty -seven 
(29%)  of  the  articles  in  JCBI  were  evaluations.  This  difference  may  be  explained  by  evaluation  articles  in 
ETR&D  being  primarily  relegated  to  the  development  section  of  the  journal.  As  above,  this  hypothesis  has 
not  been  investigated. 

The  Problem  of  Pseudoscience 

A  deeper  analysis  of  those  studies  published  in  ETR&D  and  JCBI  which  are  empirical  in  intent  and 
quantitative  in  method  yields  a  dismal  picture  of  the  quality  of  contemporary  research  in  our  field.  In  an 
earlier  article  published  in  the  now  defunct  JCBI  (Reeves,  1 993),  I  presented  an  analysis  of  five  studies 
published  in  refereed  journals  from  the  literature  on  learner  control  (Arnone  &  Grabowski,  1992;  Kinzie  & 
Sullivan,  1989;  Lopez  &  Harper-Meriniak,  1989;  McGrath,  1992;  Ross,  Morrison,  &  O'Dell,  1989).  I 
characterized  the  research  reported  in  these  articles  as  pseudoscience.  Figure  6  summarizes  the  characteristics 
of  pseudoscience  in  the  field  of  instructional  technology. 

Ironically,  the  learner  control  articles  analyzed  in  Reeves  (1993)  are  hardly  the  worst  examples  of 
pseudoscience  in  our  field.  My  analysis  of  recent  volumes  of  ETR&D  and  JCBI  indicates  that 
pseudoscience  continues  to  dominate  research  in  the  field  of  instructional  technology.  A  conservative 
review  of  the  thirty-nine  "empirical-quantitative"  studies  reported  ir  ETR&D  indicates  that  twenty-eight  of 
them  (72%)  can  be  identified  as  examples  of  pseudoscience  in  that  they  possess  two  or  more  of  the 
characteristics  in  Figure  6.  In  JCBI,  thirty-four  (61  %)  of  the  fifty-six  "empirical-quantitative"  studies 
published  during  this  period  suffer  two  or  more  signs  of  pseudoscience.  This  analysis  is  evidence  of  a 
research  malaise  of  epidemic  proportions. 
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Specification  error 

Vague  definitions  of  the  primary  independent  variables  (e.g.,  learner  control 
versus  Drosram  control). 

Lack  of  linkage  to  robust 
theory 

Little  more  than  nominal  attention  to  the  underlying  learning  and  instructional 
theories  that  are  rele  'ant  to  the  investigation. 

Inadequate  literature 
review 

Cursory  literature  review  focused  on  the  results  of  closely  related  studies  with 
little  or  no  consideration  of  alternative  findings. 

Inadequate  treatment 
implementation 

Infrequent  (usually  single)  treatment  implementation  often  averaging  less  than 
30  minutes. 

Measurement  flaws 

Precise  measurement  of  easy-to-measure  variables  (e.g.,  time);  insufficient 
effort  to  establish  the  reliability  and  validity  of  measures  of  other  variables. 

Inconsequential  outcome 
measures 

A  lack  of  intentionality  in  the  learning  context,  usually  represented  by 
outcome  measures  that  have  little  or  no  relevance  for  the  subjects  in  the  study. 

Inadequate  sample  sizes 

Small  samples  of  convenience,  e.g.,  the  ubiquitous  undergraduate  teacher 
education  orjjsychology  majors. 

Inappropriate  statistical 
analysis 

Use  of  obscure  statistical  procedures  in  an  effort  to  tease  statistically 
significant  findings  out  of  the  data. 

Meaningless  discussion 
of  results 

Rambling,  often  incoherent,  rationales  for  failing  to  find  statistically 
significant  findings. 

Figure  6.  Characteristics  of  pseudoscience  (Reeves,  1993). 

The  question  inevitably  arises  with  respect  to  how  so  many  pseudoscience  studies  get  published.  At  least 
part  of  the  answer  rests  in  the  incestuous  nature  of  the  relationships  among  the  people  conducting  these 
studies  and  the  people  charged  with  peer  review  of  these  submissions.  The  review  boards  of  these  journals 
include  many  of  the  same  people  whose  research  studies  exemplify  pseudoscience.  Not  only  does  the 
insular  nature  of  the  review  process  assure  these  researchers  of  a  venue  for  their  pseudoscience  reports,  but  it 
also  at  least  partially  explains  the  under  representation  of  alternative  approaches  of  inquiry. 

To  understand  the  steady  flow  of  pseudoscience  in  instructional  technology,  it  is  necessary  to  look  at  its 
source.  Most  of  it  emanates  from  colleges  and  schools  of  education  that  have  graduate  programs  in 
instructional  technology.  As  Kramer  (1991)  points  out  in  Ed  School  Follies,  these  inr  iutions  are  "intent 
on  proving  that  education  is  an  academic  discipline  with  its  own  subject  matter  worthy  of  a  place  alongside 
other  university  schools  and  departments  (p.  8).  The  faculty  in  these  programs  are  subject  to  the  same 
"publish  or  perish"  pressure  as  their  colleagues  in  arts  and  sciences.  They  quickly  learn  that  it  is  the 
number  of  refereed  publications  they  can  amass,  not  the  relevance  or  value  of  their  research,  that  really 
matters  when  they  come  up  for  tenure  and  promotion. 

Needless  to  say,  this  problem  is  hardly  limited  to  instructional  technology  programs.  Colleges  and  schools 
of  education  reward  pseudoscience  in  every  discipline  irom  early  childhood  education  though  vocational 
education.  A  new  report  issued  by  the  Holmes  Group  called  "Tomorrow's  Schools  of  Education,"  calls  for 
tenure  and  promotion  guidelines  to  be  revamped  so  that  professors  are  rewarded  less  for  research  and 
publication  and  more  for  work  in  the  public  schools  (Nicklin,  1995).  If  such  a  radical  shift  in  the  reward 
structure  could  be  accomplished,  I  cannot  believe  that  we  would  continue  to  conduct  pseudoscience  when  we 
could  be  rewarded  for  making  a  difference  in  the  schools  where  the  needs  are  so  great. 

Frankly,  the  likelihood  of  changing  the  reward  structure  within  universities  seems  at  best  remote. 
However,  as  instructional  technologists,  we  do  not  have  to  wait  for  such  a  change  to  occur.  Another  way 
of  increasing  the  relevance  of  instructional  technology  would  be  to  call  a  moratorium  on  our  efforts  to  find 
out  how  instructional  technology  can  effect  learning  through  empirical  research.  Instead,  we  should  turn 
our  attention  to  making  education  work  better.  It  seems  obvious  to  me  that  we  stand  a  better  chance  of 
having  a  positive  influence  on  educational  practice  if  we  engage  in  developmental  research  situated  in 
schools  with  real  problems. 

Can  reports  of  developmental  research  be  published?  Of  course!  Afer  all,  as  noted  above,  the  same  people 

ho  conduct  the  research  are  the  gatekeepers  who  determine  what  is  accepted  for  publication  in  our  most 
important  journals.  As  academics,  we  are  all  in  this  together,  and  if  we  want  to  fundamentally  change  the 
nature  of  our  "game,"  wc  can.  In  this  manner,  wc  can  still  meet  the  frustrating,  but  practical,  requirements 
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of  the  larger  academic  game  by  providing  our  instructional  technology  scholars  with  an  outlet  in  refereed 
publications,  albeit  ones  that  have  been  radically  improved  in  terms  of  goals,  methods,  and  relevancy. 

Steps  Toward  Socially  Responsible  Research 

The  relevance  of  pseudoscience  research  studies  is  a  moot  point.  Even  if  the  researchers  themselves  ascribe 
to  the  highest  ideals  of  scientific  inquiry,  research  so  flawed  has  little  relevance  for  anyone  other  than  the 
people  who  conduct  and  publish  it.  But  it  is  not  enough  to  criticize  research  in  instructional  technology  as 
characterized  by  pseudoscience  and  social  irrelevance.  Alternatives  to  the  old  ways  must  be  found.  Some 
may  demur,  believing  that  instructional  technologists  are  incapable  of  conducting  valid,  socially  relevant 
research,  and  that  they  should  stick  to  instructional  design  and  evaluation,  leaving  educational  research  to 
cognitive  psychologists  or  practitioners  better  equipped  to  conduct  it.  I  disagree.  I  think  we  can  and  will 
conduct  meaningful  research  provided  we  acknowledge  the  sterility  of  our  existing  research  base  and  build 
anew  from  a  foundation  of  sound  learning  theory  and  rededicated  concern  for  the  social  impact  of  our 
research. 

What  would  be  the  nature  of  a  new  socially  relevant  research  agenda?  Two  recent  studies  that  represent  a 
change  in  direction  toward  developmental  research  are  the  dissertation  study  conducted  by  Idit  Harel  at 
M.I.T.  (1991)  and  the  on-going  research  of  Richard  Lehrer  (1993)  and  his  associates  at  the  University  of 
Wisconsin. 

Harel's  (1991)  Instructional  Software  Design  Project  (ISDP)  represents  a  unique  effort  to  use  programming 
as  a  cognitive  tool  within  a  software  design  context.  Harel's  ISDP  combines  Papert's  "constructionist" 
theory  (1993)  with  Perkins  "knowledge  as  design"  pedagogy  (1986).  In  her  dissertation  research,  seventeen 
fourth  grade  students  used  Logo  for  a  semester  to  create  software  products  that  were  intended  to  teach 
fractions  to  third  grade  students.  Her  study  combined  quantitative,  qualitative,  and  comparative  research 
methods  to  investigate  the  effects  of  this  "learners  as  designers"  approach. 

Harel  reports  that  the  fourth  grade  students  spent  an  average  of  seventy  hours  working  on  their  software 
design  projects.  The  actual  nature  of  the  software  the  students  designed  was  open,  but  they  were  two 
requirements  for  students  in  the  program:  1)  writing  in  a  "Designer's  Notebook"  every  day,  and  2)  attending 
periodic  "Focus  Sessions"  about  software  design,  Logo  programming,  and  fractions.  A  teacher  and  the 
researcher  were  available  at  all  times  to  help  the  students  with  their  design  efforts.  Although  each  of  the 
students  produced  a  separate  software  product,  collaboration  among  the  students  was  encouraged. 

Harel  compared  the  differences  in  Logo  skills  and  fractions  knowledge  between  the  seventeen  students  in  the 
ISDP  and  thirty-four  other  students  in  two  classes  who  were  studying  Logo  and  fractions  via  "a  traditional 
teaching  method"  (p.  263).  No  significant  differences  were  found  in  pretests  among  the  three  classes.  Harel 
reports  that  "In  general,  the  17  children  of  the  experimental  class  did  better  than  the  other  34  children  on  all 
posttests  (Fractions  and  Logo)"  (p.  272).  Although  not  all  differences  were  statistically  significant,  the 
general  trend  was  quite  positive  in  terms  of  specific  learning  outcomes  as  measured  by  multiple  measures 
including  paper-and-pencil  tests,  computer  exercises,  video-taped  observations,  and  interviews. 

The  major  part  of  Harel's  (1991)  study  is  a  detailed  description  of  the  activities  and  metacognition  of  one 
student,  "Debbie,"  over  the  four  month  period  of  the  project.  Harel  wrote  that  her  detailed  analysis  of 
Debbie's  work  as  well  as  her  observations  of  other  students  indicated  that  "Throughout  ISDP,  the  students 
were  constantly  involved  in  metacognitive  acts:  learning  by  explaining,  creating,  and  discussing  knowledge 
representations,  finding  design  strategies,  and  reflecting  on  all  of  the  above"  (p.  359).  In  addition  to 
positive  cognitive  effects  in  terms  of  metacognition,  Harel  concluded  that  the  ISDP  students  acquired 
enhanced  cognitive  flexibility,  better  control  over  their  problem-solving,  and  greater  confidence  in  their 
thinking  abilities.  She  notes  however  that  the  study  did  not  include  any  direct  measures  of  thinking  skills, 
but  is  grounded  in  her  own  interpretations  of  the  students'  metacognition  and  problem-solving  processes 
based  upon  observations  and  analysis  of  documentation  such  as  their  Designer's  Notebooks. 

Lehrer  (1993)  describes  the  development,  use,  and  results  of  a  hypermedia  construction  tool  called 
Hyper\uthor  that  eighth  graders  used  to  design  their  own  lessons  about  the  American  Civil  War.  This 
approach  is  based  upon  the  cognitive  learning  theory  that  knowledge  is  a  process  of  design  and  not 
something  to  be  transmitted  from  teacher  to  student  (Perkins,  1986).  Lehrer's  students  were  engaged  in 
"hyper-composition"  by  designing  their  own  hypermedia.  In  this  mode,  learners  transform  information  into 
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dimensional  representations,  determine  what  is  important  and  what  is  not,  segment  information  into  nodes, 
link  the  information  segments  by  semantic  relationships,  and  decide  how  to  represent  ideas.  This  is  a 
highly  motivating  process  because  authorship  results  in  ownership  of  the  ideas  in  the  hypermedia 
(Jonassen,  in  press). 

Lehrer's  subjects  were  high  and  low  ability  eighth  graders  who  worked  at  the  hypermedia  construction  tasks 
for  one  class  period  of  45  minutes  each  day  over  a  period  of  several  months.  The  students  worked  in  the 
school's  media  center  where  they  had  access  to  a  color  Macintosh  computer,  scanner,  sound  digitizer, 
HyperAuthor  software,  and  print  and  non-print  resources  about  the  Civil  War.  An  instructor  was  available 
to  coach  students  in  the  conceptualization,  design,  and  production  of  hypermedia.  Students  created  programs 
reflecting  their  unique  interests  and  individual  differences.  For  example,  they  created  hypermedia  about  the 
role  of  women  in  the  Civil  War,  the  perspectives  of  slaves  toward  the  war,  and  "not-so-famous  people" 
from  that  period. 

According  to  Lehrer,  "The  most  striking  finding  was  the  degree  of  student  involvement  and  engagement"  (p. 
209).  Both  high  and  low  ability  students  became  very  task-oriented,  increasingly  so  as  they  gained  more 
autonomy  and  confidence  with  the  cognitive  tools.  At  the  end  of  the  study,  students  in  the  hypermedia 
group  and  a  control  group  of  students  who  had  studied  the  Civil  War  via  traditional  classroom  methods 
during  the  same  period  of  time  were  given  an  identical  teacher-constructed  test  of  knowledge.  No  significant 
test  differences  were  found.  Lehrer  conjectured  that  "these  measures  were  not  valid  indicators  of  the  extent 
of  learning  in  the  hypermedia  design  groups,  perhaps  because  much  of  what  students  developed  in  the  design 
context  was  not  anticipated  by  the  classroom  teacher"  (p.  218). 

However,  a  year  later,  when  students  in  the  design  and  control  groups  were  interviewed  by  an  independent 
interviewer  unconnected  with  the  previous  year's  work,  important  differences  were  found.  Students  in  the 
control  group  could  recall  almost  nothing  about  the  historical  content,  whereas  students  in  the  design  group 
displayed  elaborate  concepts  and  ideas  that  they  had  extended  to  other  areas  of  history,  Most  importantly, 
although  students  in  the  control  group  defined  history  as  the  record  of  the  facts  of  the  past,  students  in  the 
design  class  defined  history  as  a  process  of  interpreting  the  past  from  different  perspectives.  In  short,  the 
hypermedia  "design  approach  lead  to  knowledge  that  was  richer,  better  connected,  and  more  applicable  to 
subsequent  learning  and  events"  (p.  221). 

A  New  Beginning 

What  a  contrast  exists  between  the  Hard  (1991)  and  Lehrer  (1993)  studies  and  the  morass  of  pseudoscience 
endemic  in  our  field!  In  the  first  instances,  pedagogical  models  grounded  in  robust  learning  theories  have 
been  identified,  and  subsequently,  powerful  technologies  have  been  used  to  implement  these  models.  In  the 
latter,  the  power  of  various  forms  of  technology  to  instruct  is  assumed,  and  reductionist  experiments  are 
conducted  to  detect  its  effects.  In  Harel's  study,  children  with  authentic  needs  experienced  a  powerful 
learning  opportunity  over  a  period  of  months.  In  most  pseudoscience  studies,  undergraduates  earn  "extra 
credit"  for  less  than  an  hour  of  their  time  spent  using  some  form  of  mediated  "treatment"  that  has  little  or 
no  relevance  for  them. 

In  a  landmark  paper  about  educational  research,  Salomon  (1991)  describes  the  contrast  between  analytic  and 
systemic  approaches  to  research.  Salomon  claims  that  this  contrast  transcends  the  "basic  versus  applied"  or 
"quantitative  versus  qualitative"  arguments  that  so  often  dominate  debates  about  the  relevancy  of  educational 
research.  Salomon  concludes  that  the  analytic  and  systemic  approaches  are  complementary,  arguing  that 
"the  analytic  approach  capitalizes  on  precision  while  the  systemic  approach  capitalizes  on  authenticity  (p. 
16). 

While  I  agree  with  Salomon  in  theory,  the  dominance  of  pseudoscience  in  LT.  invalidates  this 
complementarity  in  practice.  The  ugly  truth  is  that  many  of  us  who  engage  in  analytic  research  approaches 
consistently  violate  many  of  the  basic  premise  >  of  this  paradigm,  especially  with  respect  to  the  testing  of 
meaningful  hypotheses  derived  from  strong  theory  (Reeves,  1993).  Although  we  may  eventually  be  able  to 
conduct  valid,  socially  responsible  analytic  studies  in  this  field,  that  time  has  not  yet  arrived.  We  need  a 
valid  body  of  systemic  (interpretivist,  postmodern,  and  developmental)  research  before  we  begin  to  have 
theoretical  foundations  strong  enough  to  pursue  an  analytical  agenda. 
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Is  instructional  technology  research  socially  responsible?  At  the  present  time,  it  is  not.  Are  we  asking  the 
wrong  questions?  For  the  most  part,  yes.  Can  wc  change  this  sad  state  of  affairs?  Of  course,  if  we  have 
the  will!  Salomon  (1991)  points  the  way.  A  major  benefit  of  systemic  research  in  education  is  that  it 
yields  new  questions  and  nurtures  the  development  of  new  theory.  The  aforementioned  moratorium  on 
analytic  studies  in  our  field  could  give  us  the  theoretical  foundations  for  a  socially  relevant  analytic  research 
agenda  early  in  the  21st  Century.  There  are  hopeful  signs  as  indicated  by  the  studies  of  Harel  (1991)  and 
Lehrer  (1993)  and  the  methodological  prescriptions  of  Neuman  (1989),  Newman  (1990),  and  Salomon 
(1991). 

I  believe  that  the  research  malaise  in  LT.  stems  in  part  from  the  "mindlessness"  that  is  endemic  in  so  much 
of  our  professional  and  personal  lives  as  we  near  the  21st  Century.  Although  some  would  attempt  to 
redirect  or  revive  the  pseudoscience  approach  to  IX  research  (cf.,  Ross  &  Morrison,  1989),  it  is  clear  that 
much  of  our  research  continues  to  suffer  from  the  same  mindless  misconceptions  and  irrelevance  identified 
by  previous  critics  (Lumsdaine,  1963;  Mielke,  1968;  Schramm,  1977;  Clark,  1983;  Salomon,  1991).  The 
social  psychologist,  Ellen  Langer,  documents  the  terrible  costs  of  mindless  behavior  in  education,  health 
care,  and  business  in  her  book  1989  book,  Mindfulness.  She  writes: 

When  we  are  behaving  mindlessly,  that  is  to  say,  relying  on  categories  drawn  in  the  past,  endpoints  to 
development  seem  fixed.  We  are  then  like  projectiles  moving  along  a  predetermined  course.  When  we  are 
mindful,  we  see  all  sorts  of  choices  and  generate  new  endpoints.  Mindful  involvement  in  each  episode  of 
development  makes  us  freer  to  map  our  own  course,  (pp.  96-97) 

The  demise  of  JCBI,  the  recurring  "influence  of  media"  debate,  and  the  prevalence  of  pseudoscience  in  our 
field  are  all  signals  that  we  need  to  become  more  mindful  about  our  research.  If  we  continue  as  before, 
mindlessly  conducting  pseudoscience,  the  obsolescence  of  our  field  per  se  is  a  likely  outcome.  Already,  the 
most  exciting  learning  and  performance  environments  are  not  coming  out  of  Departments  of  Instructional 
Technology  (cf.,  Cognition  and  Technology  Group  at  Vanderbilt,  1992;  Gery,  1995).  On  the  other  hand,  as 
Langer  emphasizes,  mindfulness  opens  up  all  kinds  of  possibilities.  Let  us  seize  this  opportunity  to  stop 
being  pawns  in  "someone  else's  costly  construction  of  reality"  (p.  28)  and  realize  that  we,  and  we  alone,  can 
assure  the  validity  and  social  relevance  of  research  in  instructional  technology. 


Background  of  Author 

THOMAS  C.  REEVES,  Ph.D.,  is  a  professor  of  instructional  technology  at  The  University  of  Georgia  where 
he  teaches  evaluation  and  multimedia  design.  Since  receiving  his  Ph.D.  at  Syracuse  University  in  1979,  he 
has  developed  and  evaluated  numerous  interactive  multimedia  programs  for  education  and  training.  He  has 
been  an  invited  speaker  in  Australia,  Brazil,  Bulgaria,  Finland,  Peru,  Russia,  South  Africa,  Switzerland, 
Taiwan,  and  the  USA.  His  research  interests  include:  1)  evaluation,  2)  mental  models,  3)  electronic 
performance  support  systems,  and  4)  instructional  technology  in  developing  countries. 

Notes 

1 .  The  belief  that  biologists  and  other  natural  scientists  don't  have  to  be  as  concerned  about  the  social 
relevance  of  their  research  as  social  scientists  is  being  tested  in  the  courts  (Leatherman,  1995).  A  female 
biology  professor  denied  tenure  at  Vassar  College  sued  on  the  grounds  that  the  research  of  the  male 
professors  who  voted  on  her  tenure  decision  was  less  important  than  hers.  The  judge  agreed  finding  that  her 
research  on  skin  differentiation  might  have  "important  implications"  for  cancer  research  whereas  the  research 
of  the  one  of  her  male  colleagues  on  spider  behavior  was  "narrow"  and  subject  to  ridicule  (p.  A 14). 

2.  Although  I  would  have  preferred  to  examine  research  publications  in  both  journals  dui.ng  an  identical 
six  year  period,  this  was  not  possible.  ETR&D  began  its  new  format  in  1989,  and  JCBI  ceased  publication 
at  the  end  of  1993. 

3.  I  am  grateful  to  Marcy  Driscoll,  Don  Ely,  Kent  Gustafson,  Mike  Hannafin,  John  Hedberg,  and  Walter 
Dick  for  their  generous  guidance  in  the  development  of  this  revised  classification  scheme.  Of  course,  I  take 
full  responsibility  for  the  flaws  that  will  no  doubt  be  revealed  in  its  organization. 

4.  ETR&D  is  a  product  of  the  integration  of  two  journals  previously  published  by  the  Association  for 
Educational  Communications  and  Technology  (Educational  Communications  and  Technology  Journal  and 
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Journal  of  Instructional  Development).  ETR&D  is  divided  into  two  sections,  a  research  section  and  a 
development  section.  This  analysis  only  considered  the  research  section  of  ETR&D. 
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Abstract 


Jn  the  tradition  of  dual  versus  single  channel  research,  tnis  field  experiment  investigated 
the  effects  of  the  inclusion  of  matched  redundant  digital  audio  on  achievement  in,  time 
spent  in  learning  and  attitude  towards  CBI  delivered  mathematics  at  the  high  school  level. 
Differential  effects  on  students  of  varying  entry  learning  math  performance  was  also 
investigated.  Mathematics  10  subjects  (N=82)  were  assigned  to  the  two  main  treatment 
conditions  (CBI_audio,  CBI_text)  by  stratified  match  pairs  within  three  existing  classes. 
Both  treatment  groups  completed  the  same  three  lessons  of  one  unit  of  MathTech  10. 
MathTech  10  is  a  course  in  the  Alberta  curriculum  which  is  delivered  by  CBI,  contains 
1 10  lessons  and  incorporates  16  self-diagnostic  exams.  The  three  selected  lesson  files  for 
the  treatment  condition  were  modified  by  adding  redundant  audio  (male  voice  instructions), 
except  for  the  exams  and  those  portions  of  the  lesson  where  random  generation  were  used. 
Analysis  of  scores  of  a  math  achievement  test  measuring  the  objectives  of  the  lessons 
revealed  that  assignment  to  the  two  CBI  delivery  modes  did  not  have  a  significant  effect 
on  the  overall  comprehension  and  learning  mastery  of  the  related  mathematical  materials. 
No  significant  differences  between  the  lower  ability  students  across  the  two  treatment 
groups  were  observed.  Several  different  time  measures  were  addressed  in  the  study. 
Overall  study  times  were  equivalent  for  main  treatment  groups  but  higher  ability  dual 
channel  students  took  more  time  overall  than  their  control  group  counterparts.  Regarding 
time  to  complete  the  practice  questions  built  into  each  lesson,  the  higher  abi!i-v  dual 
channel  students  took  less  time  to  complete  the  questions  with  equivalent  achievement 
scores,  implying  greater  learning  efficiency.  Overall  there  were  no  significant  attitude 
differences  between  the  two  main  treatment  groups.  In  the  dual  channel  audio  treatment 
condition,  lower  ability  students  were  generally  more  positive  about  learning  than  the 
higher  ability  students.  Additional  attitude  differences  were  found.  The  results  suggest 
that  redundant  audio,  added  to  Computer  Based  Instruction  to  teach  high  school 
mathematics,  has  little  effect  on  achievement  but  does  affect  learning  time,  and  attitude 
formation  of  students  of  differing  learning  ability. 


The  emergence  of  more  powerful  computer  hardware  and  software  development  tools  make  it  technically 
feasible  to  incorporate  digital  audio  speech  into  interactive  Computer  Based  Instruction  (CBI).  While 
technically  feasible,  little  actual  research  has  been  undertaken  with  respect  to  investigating  the  educational 
effects  of  incorporating  digital  audio  speech  on  the  learning  process. 

Digital  audio  speech  research  has  narrowly  focused  on  the  mechanics  of  incorporating  digital  audio  speech 
into  existing  text-based  applications,  the  "how  to"  of  digital  voice  integration.  Contemporary  research 
assumes  the  relative  educational  effectiveness  and  desirability  of  the  inclusion  of  digital  audio  speech  in 
text-based  applications.  From  a  hardware  and  software  perspective,  it  focuses  on  discovering  the  best 
techniques  of  incorporating  digital  audio  speech  into  text-based  applications.  Areas  that  have  received  the 
most  attention  for  the  inclusion  of  digital  audio  speech  are  second  language  training,  distance  education, 
adult  literacy,  special  education,  early  childhood  education  and  reading  programs. 

Little  research  to  date  has  investigated  the  relative  benefits  and  educational  effectiveness  of  the  inclusion  of 
digital  audio  speech  in  CBI  applications,  the  "why"  of  audio  integration.  Furthermore,  very  little 
information  is  available  regarding  the  educational  effect,  benefits  and  desirability  of  the  inclusion  of  digital 
audio  speech  in  CBI  in  core  subject  areas  in  mainstream  educational  programs,  especially  at  the  junior  and 
senior  high  school  levels.  Specifically,  the  research  has  not  addressed  whether  the  inclusion/exclusion  of 
digital  audio  speech  constitutes  a  significant  user  variable  in  the  enhancement  of  student  achievement  and 
attitudes  towards  CBI  Mathematics  learning  at  the  senior  high  school  level. 

Research  on  Single  and  Dual  Channel  Learning  in  the  Context  of  CBI 

Although  substantial  research  has  been  conducted  on  the  effects  of  speech  and  audio,  the  results  as  yet  do 
not  provide  clear  direction.  This  section  summarizes  some  of  the  key  findings  and  conclusions. 
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One  of  the  three  stages  of  our  short  term  memory  seems  devoted  to  image  retention  (iconic  memory  for 
visuals,  echoic  memory  for  auditory  and  haptic  for  touch  and  senses)  (Dempster,  1985).  People  can 
discriminate  among  4  sounds  at  a  time  but  the  short  term  echoic  storage  lasts  only  2  seconds  before 
accuracy  of  recall  begins  to  decline  (Darwin,  Turvey  &  Crowder,  1972). 

Studies  have  examined  learning  from  single  and  dual  channel  sources.  McLuhan  observed  in  1969  that  when 
information  is  simultaneous  from  all  directions  at  once,  the  culture  is  auditory  and  tribal,  regardless  of  its 
pas  or  its  concepts. 

Hartman  (1961)  identified  4  kinds  of  multichannel  information:  redundant  (e.g.,  identical  audio  and  text; 
related  (pictorial  representation  and  verbal  description);  unrelated  as  in  a  picture  accompanied  by  a  different 
spoken  word;  and  contradictory  (simultaneous  presentation  of  a  picture  of  a  man  and  the  spoken  word 
'women').  Baecker  &  Buxton  (1987)  identified  several  uses  for  sound;  communicating,  seizing  attention, 
enhancing  visual  stimuli,  and  relieving  overload  on  the  learner's  information  processing  and  constructivist 
learning  system.  As  a  practical  example  in  video,  Hanson  (1989)  recommended  using  redundant  audio  and 
video  if  information  is  to  be  learned,  otherwise  use  non-redundant  audio  to  direct  attention  to  the  video 
channel. 

Hartman  (1961)  reviewed  nine  studies  comparing  simultaneous  audio-print  presentations  with  audio  alone 
or  print  alone  and  found  seven  of  those  supported  simultaneous  presentation  with  the  remaining  being 
equal.  Hsia  (1968)  concluded  that  communication  efficiency  and  dependability  were  higher  when  audio  and 
visual  channels  were  present  compared  with  audio  and  visual  alone. 

Hsia  (1969)  argued  that  simultaneous  audio  visual  presentation  of  identical  or  highly  redundant  information 
should  increase  the  performance  of  low  intelligence  learners  to  that  of  high  intelligence  learners.  Kozma 
(1991)  suggested  that  students  who  are  more  knowledgeable  about  the  subject  can  proceed  faster  and  vice 
versa;  these  suggest  research  on  interactions  between  aptitudes  of  learners  and  multichannel  treatments. 

Channel  research  involving  audio  is  far  from  conclusive.  Barron  (1991)  for  example,  found  no  difference 
between  students  who  learned  via  CBI  with  and  without  audio.  In  terms  of  learning  time,  Laddaga,  Levine 
and  Suppes  (1981)  found  students  took  the  same  amount  of  time  in  CBI  audio  as  CBI  non-audio  but 
perceived  the  audio  as  taking  longer.  When  students  were  allowed  to  choose  audio  vs  non-  audio  CBI 
instruction,  about  half  chose  audio. 

The  Research  Study 

The  purpose  of  this  study  was  to  determine  the  effects  of  CBI  and  redundant  audio,  as  measured  by 
achievement,  attitude  and  time  spent  in  learning  10th  grade  math  concepts.  In  addition  to  comparing 
CBI_audio  and  CBI_text,  differential  effects  on  students  of  varying  entry  learning  math  performance  was 
investigated.  It  was  expected  that  the  treatments  would  effect  students  of  different  learning  histories 
differently. 

Materials  and  Procedures 

Math7ec/i  10  is  a  course  in  the  Alberta  curriculum  which  is  delivered  by  CBI,  contains  1 10  lessons  and  16 
exams,  and  is  suitable  for  distance  delivery  and  individualized  instruction. 

Three  lessons  on  Equalities  and  Inequalities  developed  in  Authorware  Professional  1.6  were  selected  and 
utilized  in  their  original  screen  design,  instructional  content  and  feedback.  CBI_audio  was  modified  by 
adding  redundant  audio,  except  for  the  exams  and  those  portions  of  the  lesson  where  random  generation  was 
used.  Each  lesson  includes  instruction  and  a  practice  section. 

The  male  voice  audio  of  on  screen  text  instructions  and  student  performance  feedback  was  digitized  using  a 
MacRecorder.  After  extensive  pretesting  of  various  combinations  of  sampling  and  compression  rates  in 
conjunction  with  the  specific  hardware/software  parameters  of  the  Macintosh  Classic  and  Authorware 
Professional  environment,  the  1 1  Khz  sampling  rate  was  selected.  This  sampling  rate  offered  the  best 
available  quality  voice  recording  given  the  test-site  parameters  -  4  MB  RAM,  20  MB  Macintosh  Classic 
Hard  Drive,  512  K  cache.  As  noted  by  Barron  (1991),  the  1 1  Khz  sampling  rate  exceeds  acceptable  industry 
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standards  for  voice  recording  at  the  time  of  the  study.  Due  to  the  CBI_audio  lessons'  large  size  and  demand 
on  processing  speed,  all  lessons  (CBI_audio  and  CBI_text)  were  run  on  30  stand-  alone  Macintosh  Classic 
learning  stations  rather  than  delivered  through  the  school's  MacJanet  network.  Program  modifications 
resulted  in  diffea  ...ial  file  sizes  (CBI_audio  =  17  MB,  CBI_text  =  1.2MB).  Audio  compression  was  not 
used  in  this  study  due  to  technological  limitations.  Data  were  collected  in  the  Fall  of  1993  from  a  large 
suburban  high  school 

The  lessons  took  335  minutes  to  complete  over  5  days.  Students  (N  =  82)  from  the  three  classes  (2 
teachers)  in  a  single  high  school  were  randomly  assigned  in  stratified  matched  pairs  to  treatment 
(CBI_audio)  or  control  (CBI_text). 

The  field  experiment  used  a  two-factorial  design  (ability  X  treatment)  across  three  dependent  variables 
(achievement  in,  time  spent  in  learning  and  expressed  attitude  towards  CBI  mathematics).  The  two  levels  of 
entry  level  math  abilities  were  measured  by  scores  in  the  previous  year  of  mathematics  (higher  ability 
students  mean  Grade  9  final  mark  =  86.30,  lower  ability  students  mean  Grade  9  final  mark  =  61.14). 

Data  Sources 

The  dependent  variable  of  achievement  was  measured  by  a  series  of  content  examinations  and  a  25  item  Unit 
exam  targeted  to  the  objectives  of  the  three  lessons.  Time  spent  in  learning  was  assessed  using  the  built-in 
clock  function  of  the  computer  and  included  completion  of  instruction  module,  completion  of  practice 
questions  and  total  connect  time.  Attitude  was  assessed  through  a  42  item  Likert-  type  instrument  designed 
to  identify  attitude  towards  learning  mathematics  by  computer. 

Results-  Achievement 

Analysis  of  scores  of  a  math  achievement  test  measuring  the  objectives  of  the  lessons  revealed  no  difference 
between  the  two  treatment  groups  (Table  1)  and  no  difference  between  lower  ability  students  across  the  two 
treatment  groups  (Table  2). 


Tabla  1 
t-teat    for    Paired  Samples 

on    Final    Exam  Scores 


No  of  Pairs: 

41     df:   40  t 

-value (0.95)  : 

-.06 

Group : 

Mean: 

SD: 

SE: 

CBI_audio 

64.83 

18.17 

2.84 

CBI_text 

65  .  00 

17  .  50 

2  .73 

Tabla 

2 

ANOVA 

-    Lower    Ability  Students 

Final  Exam 

Scores 

Source              df       Sum  SQ            Mean  SQ      F  P 

Between:     1              107.3              107.3  .45  .51 

Within:      40  9547.0  238.7 
Total:       41  9654.3 

Group:  Mean:               SD:  SE:  Range 

CBI_audio  52.58  17.07  3.73  24.5-84.3 

CBI  text  55.78  13.64  2.98  32.4-80.6 
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Results-Time 


Several  different  time  measures  were  addressed  in  the  study.  Overall  study  times  were  equivalent  for 
CBl_audio  and  CBl_text  (Table  3)  but  the  higher  ability  audio  students  took  more  time  (22.3%)  overall 
(Table  4).  Regarding  time  to  complete  the  practice  questions  built  into  each  lesson,  the  CBI_audio  students 
took  less  time  (25.9%)  than  the  CBIJext  group  (with  equivalent  achievement  scores),  implying  greater 
learning  efficiency  for  CBI_audio  condition  than  for  CBIJext  (Table  5). 


Table  3 
t -test    for    Paired  Samples 
on    Time    Spent    in    Learning  (Seconds) 


No  of  Pairs: 

41     df:    40  t 

-value (1.67) :  1.41 

Group : 

Mean : 

SD :                    SE : 

CBI._audio 
CBI_text 

4421.7 
4064. 8 

1370-2  214 
1460.2  228 

ANOVA 
on  Time 

Table  4 
-Higher    Ability  Students 
Spent    in    Learning  (Seconds) 

Source 

df       Sura  SQ 

Mean  SQ       F  P 

Between: 
Within : 
Total : 

1        5. 69 

38  50.00 

39  55.72 

5.69       4.32  .045 
1.32 

Group : 

Mean : 

SD:  SE: 

CBI_audio 
CBI_text 

4126 
3372 

1302  291 
969  217 

Note:  SuraSq 

&  Mean  SQ  X  1, 000, 000 

Table  5 

ANOVA-Pract ice    Exercise    Time  (Sec) 

Source 

df      Sura  SQ 

Mean  SQ      F  P 

Between : 
Within : 
Total : 

1  6.16 

80  81.39 

81  87.55 

6.16       6.05  .02 
10.17 

Group : 

Mean:   SD:  SE: 

CBI„audio 
CBI_text 

2115     777  121 
2664     1196  187 

Note:  SimSq 

&  Mean  SQ  X  1, 000, 000 
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Results-Attitude 


In  the  CBI_audic  condition,  lower  ability  students  were  more  positive  about  learning  (13%)  than  the  higher 
ability  students  (Table  6).  Overall  there  were  no  attitude  differences  between  the  two  treatment  groups 
(Table  7).  Additional  attitude  differences  were  found. 


Table  6 

AMOVA    -    Lower    /    Higher    Ability    Students    Within  CBI_audio 


Source 

df 

Sum  SQ 

Mean  SQ  F 

F 

Between : 

1 

542.3 

542.3  5.15 

.03 

Within: 

39 

4105.5 

105.3 

Total : 

40 

4647.8 

Group : 

Mean: 

SD :       SE : 

Range 

CBI_audio 

54.65 

10.35  2.31 

35.4-74 

9 

CBI_text 

61.93 

10.18  2.22 

36.7-81 

4 

Table  7 
t-test    for    Paired  Samples 
on    Expressed    Attitude  Scores 


No  of  Pairs:  41     df:   40  t~value ( 0 . 75 ) :  .32 

Group:  Mean:   SD:  SE: 

CBI_audio  58.38  10.78  1.68 

CBI  text  57.71     8.79  1.37 


Educational  Importance 
of  the  Study 

The  results  suggest  that  redundant  audio,  added  to  CBI  to  teach  high  school  mathematics,  has  little  effect  on 
achievement  but  does  affect  learning  time  and  attitude  formation  of  students  of  different  learning  ability.  In 
light  of  the  energy  and  effort  required  to  incorporate  audio  into  general  instruction,  the  file  space  required 
(given  current  levels  of  file  compression)  and  current  speeds  of  processing  and  delivery  of  instruction  to 
students,  one  can  legitimately  question  the  payoff  in  terms  of  achievement,  attitude,  or  learning  time.  The 
value  of  audio  in  terms  of  securing  a  publisher  for  courseware  should  be  investigated  as  a  separate  issue. 
Further  hypothesis  and  implications  are  discussed  in  the  full  presentation. 

Suggestions  for  further  research  include  an  examination  of  the  cost-effectiveness  with  respect  to  developing 
and  delivering  audio  relative  to  achievement,  instructional  time  and  a.titude.  The  sex  of  the  voicing  was 
controlled-one  could  investigate  effects  of  different  voicings,  such  as  pee.  or  authoritative  voice  crossed 
with  gender.  Finally,  the  use  of  non-redundant  audio  should  be  invec*i^ted. 


ERIC 


476 


REFERENCES 

Baecker  &  Buxton  (1987).  Readings  in  Human-Computer  Interaction:  A  Multidisciplinary  Approach.  Los 
Altos,  CA:  Morgan  Kaufmann. 

Barron ,  A.  E.  ( 1 99 1 ).  The  effectiveness  of  digital  audio  in  computer-based  training.  Unpublished  doctoral 
thesis,  University  of  Central  Florida. 

Darwin,  C.  J.,  Turvcy,  M.  T.,  &  Crowder,  R.  G.  (1972).  An  auditory  analogue  of  the  Sperling  partial 
report  procedures:  Evidence  for  brief  auditory  storage.  Cognitive  Psychology,  3,  255-267. 

Dempster,  F.  N.  (1985).  Short  term  memory  development.  In  C.  J.  Brainerd  &  M.  Pressley  (Eds.).  Basic 
process  in  memory  development.  NY:  Springer- Verlag  208-248. 

Hanson,  L.  (1989,  October).  Multichannel  research  applied  to  television  production:  A  review  and 
synthesis  of  the  literature.  Educational  Technology,  15-19. 

Hartman,  F.  (1961a)  .  Single  and  multiple  channel  communication:  A  review  of  research  and  a  proposed 
model.  Audio  Visual  Communication  Review  8,  235-  262. 

Hartman,  F.  (1961b,  November  /December) .  Single  and  multiple  channel  communications:  A  review  of 
research  and  a  proposed  model.  AV  Communication  Review  9,  (6),  235-262. 

Hsia,  H.  J.  (1968a).  On  channel  effectiveness.  AV  Communication  review,  16,  245-267. 

Hsia,  H.  J.  (1968b,  December)  .  Output,  error,  equivocation  and  recalled  information  in  auditory,  visual 
and  audiovisual  information  processing.  Journal  of  Communication,  18,  325-353. 

Kozma,  R.B.  (1991,  Summer) .  Learning  with  Media.  Review  of  Educational  Researcn.  61,  (2),  179-21 1. 

Laddaga,  R.,  Levine,  A.,  &  Suppes,  P.  (19981).  Studies  of  student  preferences  for  computer-assisted 
instruction  with  audio.  In  P.  Suppes  (Ed.),  University-  Level  Computer-Assisted  Instruction  at 
Stanford:  1968-1981.  Stanford,  CA:  Stanford  University. 


ERIC 


4  * 


477 


Title: 


The  Relationships  Among  Measures  of  Intrinsic  Motivation, 
Instructional  Design,  and  Learning 
in  Computer-Based  Instruction 


Author: 

Randy  Rezabek,  Ph.D. 
The  University  of  Oklahoma 

1220  East  Brooks  St. 
Norman,  OK  73071 
(405)  329-7771 
Internet:  dd8244@uoknor.edu 


4S2 

er|c 

ummmmmm 


Why  would  a  person  want  to  learn?  This  is  one  of  the  central  questions  of  education.  Many 
arguments  have  been  generated  as  to  why  learning  benefits  both  the  individual  and  their  society.  But  these 
rationales  focus  on  the  outcomes  of  learning,  not  the  process  itself.  For  instructional  designers,  whose  task 
it  is  to  create  learning  environments,  the  key  question  is  "what  variables  encourage  learning  for  its  own 

sake?" 

The  study  of  motivation  has  long  been  a  neglected  area  in  instructional  technology.  The  emphasis 
on  promoting  effectiveness  and  efficiency  in  instructional  design  often  excludes  concerns  about  the  appeal  of 
instruction.  Traditionally  handicapped  by  the  lack  of  theory  and  lack  of  measurements  dealing  with 
motivation,  instructional  designers  have  assumed  that  good  quality  instruction  will  in  itself  be  motivating 
(Keller,  1983).  However,  motivation  is  such  a  crucial  issue  in  education  that  simple  assumptions  are  not 
sufficient. 

Our  culture  tends  to  emphasize  extrinsic  motivation,  that  is,  the  use  of  externally  supplied  rewards 
and  punishments  as  a  means  of  controlling  behavior.  However,  the  use  of  an  extrinsic  reward  has  a  hidden 
cost  in  that  it  has  a  negative  effect  on  an  individual's  intrinsic  motivation,  that  is,  motivational  stat 
which  are  internally  supplied  or  generated  (Deci  &  Ryan,  1985).  With  an  intrinsic  motivational  str,  _y, 
the  reward  arises  out  of  direct  involvement  with  an  ongoing  activity,  which  in  turn,  increases  the 
participant's  ability  to  grow  in  terms  of  personal  complexity  (Csikszentmihalyi,  1978). 

The  intent  of  this  study  was  to  explore  the  intrinsic  aspects  of  motivation,  and  to  see  if  the  design 
of  instruction  could  positively  affect  learners'  levels  of  intrinsic  motivation  toward  the  subject  matter. 

Statement  of  the  Problem 

This  research  concerned  the  relationship  between  intrinsic  motivation,  differing  instructional 
designs,  and  learning  in  computer-based  instruction.  Specifically,  it  addressed  the  following  questions: 

1 .  Will  different  computer-based  instructional  treatments  which  have  been  designed  to  reflect 

principles  of  intrinsic  motivation  to  varying  degrees  produce  measurably  different  levels 
of  intrinsic  motivation  in  learners? 

2.  Can  the  data  from  this  study  be  explained  in  terms  of  the  flow  model? 

3.  Are  higher  levels  of  intrinsic  motivation  associated  with  higher  learning  scores? 

4.  Are  prior  learning  and  prior  interest  related  to  different  levels  of  intrinsic  motivation? 

Theoretical  Framework 

The  primary  theoretical  framework  for  this  study  is  the  work  of  Mihaly  Csikszentmihalyi  (1975, 
1990,  i  993;  Csikszentmihalyi  &  Csikszentmihalyi,  198°'  which  is  generally  described  as  "flow  theory." 
Csikszentmihalyi's  work  deals  with  identifying  and  defining  the  structural  characteristics  of  "optimal" 
experiences,  that  is,  experiences  which  are  perceived  to  be  challenging,  exhilarating,  absorbing,  and  fun. 

The  flow  phenomenon  is  best  described  in  terms  of  its  two  parameters,  challenge  and  skills.  When 
these  two  factors  are  in  balance,  people  feel  that  they  have  the  skills  necessary  to  achieve  a  challenging,  but 
realistic,  goal  (Csikszentmihalyi,  1982).  A  challenge  is  perceived  as  an  activity  that  has  clear  goals  and 
requires  an  investment  of 

effort  in  order  for  the  goal  to  be  achieved.  Skills  are  those  efforts  that  an  individual  needs  to  apply  towards 
the  achievement  of  the  goal.  Implicit  in  this  model  is  the  characteristic  of  feedback,  which  allows  an 
individual  to  track  his  or  her  efforts  in  achieving  the  goal. 

Csikszentmihalyi  (1993)  states  that  flow-like  activities  have  four  main  characteristics: 
(1)  they  have  concrete  goals  and  manageable  rules,  (2)  they  make  it  possible  to 
adjust  opportunities  for  action  to  our  capabilities,  (3)  they  provide  clear  information 
about  how  well  we  are  doing,  .Mid  (4)  they  screen  out  distractions  and  make 
concentration  possible  (p.  xiv). 

Flow  experiences  are  informative  in  nature,  that  is,  they  are  clearly  bounded  by  rules  and  goals,  yet 
are  flexible  enough  to  allow  an  individual  to  exert  effort  and  see  the  results  of  those  efforts. 

The  emotional  effects  of  attempting  to  achieve  any  goal  can  be  classified  according  to  the  ratio  of 
challenge  to  skill  an  individual  applies  to  the  task.  These  ratios  are  termed  "channels"  and  are  grouped  into 
classes  and  defined  by  Csikszentmihalyi  as:  flow,  in  which  the  level  of  challenge  is  equal  or  slightly  above 
the  level  of  skill  required  to  achieve  the  task;  anxiety,  in  which  the  level  of  challenge  is  greater  than  the 
amount  of  skill  necessary  for  the  task;  apathy,  in  which  both  the  level  of  challenge  and  the  level  of  skill  is 
low;  and  boredom,  in  which  the  level  of  challenge  is  lower  than  the  amount  of  skill  an  individual  can  bring 
to  the  task  (Csikszentmihalyi  &  Csikszentmihalyi,  1988). 
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Flow  is  perceived  as  a  positive  experience  because  the  mastery  of  an  achievable  challenge  stretches 
an  individual's  capabilities  by  promoting  the  development  of  new  skills  and  increasing  self-esteem  and 
personal  complexly  (Csikszentmihalyi  &  LeFevre,  1989).  In  addition,  remaining  in  the  state  of  flow 
requires  further  increases  in  the  complexity  of  the  challenge,  thereby  promoting  growth. 

While  :n  a  state  of  flow,  people  often  report  feeling  more  active,  alert,  concentrated,  happy, 
satisfied,  and  creative  (Csikszentmihalyi  &  LeFevre,  1989).  Flow  produces  a  sense  of  focused  concentration 
described  as  the  "merging  of  activity  and  awareness"  which  Csikszentmihalyi  claims  "produces  harmony 
within  the  self  (Csikszentminalyi  &  Csikszentmihalyi,  1988). 

The  traditional  approach  to  studying  flow  is  through  the  use  of  the  Experience  Sampling 
Methodology.  The  ELM  randomly  samples  the  day-to-day  activities  of  subjects  over  a  period  of  time, 
usually  one  week.  These  experiences  are  captured  in  a  series  of  open-ended  questions  relating  to  what  the 
person  was  doing  and  Likert-scale  items  which  relate  to  the  individual's  motivational  state  which  can  then 
be  classified  and  quantitatively  analyzed.  This  methodology  has  been  used  with  a  wide  variety  of 
populations,  ranging  from  adolescents  to  the  elderly,  and  in  a  number  of  cross-cultural  settings  (Sato,  1988; 
Allison  &  Duncan,  1988;  Han,  1988;  Carli,  Delia  Fave,  &  Massimini,  1988).  These  studies  confirm  flow 
theory's  major  hypothesis:  that  which  makes  an  experience  enjoyable  and  motivating  is  the  balance  between 
the  challenge  of  the  activity  and  the  skills  of  the  individual. 

Although  flow  activities  can  take  many  forms,  and  to  a  certain  degree  depend  upon  individual 
interests  and  tastes,  the  theory  shows  that  manipulating  the  structural  variables  of  challenge,  skill,  and 
feedback  can  increase  the  likelihood  that  an  individual  will  find  an  activity  to  be  motivating.  Within  the 
field  of  computer  software,  video  games  are  a  classic  example  of  how  structural  factors  in  the  design  of  the 
program  can  affect  the  intensity  of  use.  Malone's  research  (1981a,  1981b)  indicates  that  the  two  factors  that 
most  clearly  distinguish  popular  video  games  are  the  ability  of  the  game  to  provide  a  clear  goal  and  to 
provide  feedback  in  the  form  of  a  score.  From  the  perspective  of  flow  theory,  the  goals  of  these  games  are 
interpreted  as  an  achievable  challenge  while  the  scoring  mechanism  provides  immediate  information  to  the 
user  as  to  their  skill  level. 

Materials  for  the  Study 

The  instructional  materials  for  this  study  consists  of  three  computer-assisted  instructional 
programs  on  the  topic  of  the  relationship  of  aperture  and  shutter  speed  to  exposure,  depth-of-field,  and  image 
blur.  These  materials  were  designed  according  to  a  set  of  prescriptions  based  upon  the  challenge,  skills,  and 
feedback  structure  of  flow  theory,  with  additional  specific  recommendations  derived  from  the  work  of  Keller 
(1983,  1987)  and  Vallerand,  Deci  &  Ryan  (1987)  (as  cited  in  Rezabek,  1994). 

Treatment  #1  utilized  a  linear  design  in  which  subjects  were  exposed  to  a  fixed  sequence  of  topics. 
This  treatment  contained  embedded  questions  with  feedback,  illustrations  and  animation  in  support  of  the 
instructional  material.  It  was  hypothesized  that  this  treatment  would  be  the  least  motivating  of  the  three  due 
to  its  inability  to  vary  the  challenge  level.  Treatment  #2  contained  the  same  content  material  but  was 
organized  into  a  "hypermedia"  style  structure  which  gave  the  subjects  a  choice  over  the  sequencing  of  the 
instruction.  It  was  hypothesized  that  this  treatment  would  be  somewhat  more  motivating  due  to  the 
increased  use  of  learner  control  over  the  material.  Treatment  #3  utilized  a  series  of  camera  simulators  with 
embedded  instructional  sequences  for  both  presentation  of  the  material  and  opportunities  for  practice.  This 
treatment  provided  feedback  in  the  form  of  photographs  whose  characteristics  varied  according  to  the 
settings  of  the  simulators.  In  addition.  Treatment  #3  utilized  a  "Jeopardy™"  style  game  at  the  end  of 
instruction  in  lieu  of  embedded  questions.  This  treatment  was  designed  to  be  the  most  motivating  of  the 
three,  incorporating  a  varying  challenge  level,  a  high  degree  of  learner  control,  and  instantaneous  feedback 
which  was  natural  to  the  task. 

A  conventional  multiple  choice  test  was  developed  to  act  as  both  a  pre-test  and  a  post-test.  A  prior 
interest  survey  form  was  developed  for  the  purpose  of  gathering  sufficient  data  to  control  for  the  impact  of 
prior-interest  on  intrinsic  motivation.  A  number  of  Likert-scale  type  questions  were  utilized  that 
emphasized  interest,  rather  than  competence,  in  the  subject  matter. 

Measures  of  intrinsic  motivation  were  gathered  through  the  use  of  the  Experience  Sampling  Form 
(ESF)  and  the  Intrinsic  Motivation  Inventory  (IMI).  The  Experience  Sampling  Form  modified  for  use  in 
this  study  was  closely  modeled  after  the  example  provided  in  Csikszentmihalyi  &  Larson  (1987).  The 
specific  data  pertaining  to  the  clusters  of  dependent  variables  in  this  study  (affect,  activation,  cognitive 
efficiency,  and  motivation)  were  provided  by  utilizing  a  number  of  Likert  scales  measuring  these 
dimensions  of  the  respondents  experience. 

A  second  measure  of  intrinsic  motivation  was  provided  through  the  use  of  the  Intrinsic  Motivation 
Inventory,  a  series  of  18  Likert  scale  items  designed  to  elicit  subjects'  responses  along  four  underlying 


dimensions  of  interest-enjoyment,  perceived  competence,  effort,  and  pressure-tension.  The  IMI  used  in  this 
study  was  closely  modeled  after  the  example  provided  in  McAuley,  Duncan  &Tammen  (1989).  This 
second  scale  was  used  to  provide  additional  data  to  support  the  reliability  and  validity  of  the  measures  of 
intrinsic  motivation  used  in  this  study. 

Experimental  Design 

The  research  design  employed  as  the  primary  dependent  variables  the  Likert  scale  questions  of  ESM 
and  the  IMI  which  were  clustered  into  their  appropriate  subdimensions.  The  independent  variables  consisted 
of  the  three  computerized  instructional  treatments,  1,  2,  and  3,  which  were  designed  to  create  predicted 
effects  of  low,  intermediate,  or  high  intrinsic  motivation.  In  addition,  the  design  employed  a  correlational 
analysis  of  the  relationship  between  intrinsic  motivation  and  learning,  and  explored  the  effects  of  prior- 
learning  and  prior-interest  on  levels  of  intrinsic  motivation.  Post  test  scores  were  used  to  measure  the 
dependent  variable  of  learning,  and  pre-test  scores  along  with  the  prior  interest  survey  was  used  as  measures 
of  prior  learning  and  prior  interest. 

Procedures 

Approximately  120  subjects  were  randomly  assigned  to  the  three  treatment  groups.  Subjects  in 
each  treatment  group  interacted  with  a  computer-based  instructional  program  designed  to  teach  the  topic  of 
the  relationship  of  aperture  and  shutter  speed  to  exposure,  depth-of-field,  and  image  blur  in  the  making  of  a 
photograph.  Each  treatment  taught  the  same  objectives  but  varied  as  to  the  instructional  design  utilized,  as 
described  previously. 

A  few  days  prior  to  the  experiment,  the  subjects  took  a  pre-test  and  completed  the  prior-interest 
survey  form.  The  pre-test  and  any  necessary  follow  on  interviews  were  used  to  screen  out  any  subjects  that 
had  a  high  level  of  subject  knowledge  to  ensure  a  normal  distribution  of  prior-knowledge.  Prior-interest 
scores  were  utilized  later  as  part  of  the  data  analysis  phase. 

Immediately  following  the  delivery  of  the  instruction  in  the  treatments,  subjects  were  asked  to  fill 
out  the  Experience  Sampling  Form,  designed  to  capture  a  quantitative  measure  of  their  level  of  intrinsic 
motivation  experienced  during  the  treatment.  In  addition,  they  were  also  asked  to  fill  out  the  Intrinsic 
Motivational  Inventory  to  provide  additional  data  for  cross  checking  the  validity  of  the  intrinsic  motivation 
measures.  Following  this,  the  subjects  completed  the  post-test. 

This  collection  procedure  yielded  the  following  types  of  data:  pre-test,  post-test,  prior-interest,  and 
two  separate  measures  of  intrinsic  motivation  for  each  subject  in  each  of  the  treatments. 

Results 

The  first  hypothesis  studied  the  effects  of  differing  instructional  treatments  on  subjects'  levels  of 
intrinsic  motivation.  It  was  expected  that  treatment  #3,  which  utilized  the  greatest  number  of  prescriptions, 
would  have  the  strongest  effect  on  intrinsic  motivation. 

Data  for  the  three  treatments.were  analyzed  by  using  Dunn's  planned  comparison  procedure  in  order 
to  find  differences  oetween  the  groups  on  measures  of  intrinsic  motivation.  Only  owz  comparison,  between 
treatments  2  and  3  on  the  Perceived  Competence  subdimension  of  the  IMI,  was  found  to  be  significant  (see 
table  1).  The  comparison  of  this  subdimension,  which  measures  respondents'  sense  of  competence,  skill  and 
confidence  toward  the  activity,  showed  that  the  mean  scores  for  the  treatment  #2  group  were  greater  than  the 
mean  scores  for  treatment  #3,  counter  to  the  predicted  direction.  Since  the  distinguishing  characteristics  of 
treatment  #2,  the  non-linear  hypermedia  structure,  was  also  present  in  treatment  #3,  it  is  possible  that  the 
significance  of  this  comparison  is  a  statistical  fluke,  and  would  disappear  in  subsequent  replications  of  the 
study.  For  the  ESF  data,  no  comparisons  were  found  to  be  significant.  The  hypothesized  superiority  of 
treatment  #3  over  the  other  treatments  on  the  other  measures  of  intrinsic  motivation  was  not  supported  by 
the  data. 

The  second  part  of  the  study  looked  at  an  aggregate  of  the  data  to  explore  the  validity  of  flow 
theory  within  an  instructional  context,  and  to  establish  a  link  between  intrinsic  motivation  and 
achievement.  A  multivariate  analysis  of  variance  (MANOVA)  was  used  to  determine  if  subjects  who  fit  the 
"flow"  condition  (both  challenge  and  skill  reported  in  the  upper  end  of  the  scale)  differed  from  subjects  who 
reported  a  "no  flow"  condition. 
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TABLE  1: 

 I  MI  Planned  Comparisons  -  Differences  Among  Means 

Interest- Enjoyment  Subdimension: 


Treatment 

2 

1  3 

.115  .229 

2  (x  =  5.49) 

.115 

1  (x  =  5.38) 

3  (x  =  5.26) 

Perceived  Competence  Subdimension: 


Treatment 

1 

2  3 

-.383  .736 

1  (x  =  5.50) 

1.119* 

2(x  =  5.88) 

3  (x  =  4.77) 

Effort-Importance  Subdimension: 


Treatment 

2 

1 

3 

-.457 

.191 

2(x  =  5.38) 

.648 

1  (x  =  5.84) 

3  fit  c  5.19) 

Tension-Pressure  Subdimension: 


Treatment 

1 

2  3 

-.303  .787 

1  (x  =6.1 1) 

1.091 

2  (x  =  6.41) 

3  fx  =  5.32) 

*  Significant  at  the  0.05  level  

For  the  Experience  Sampling  Form,  this  distinction  was  found  in  two  of  the  four  subdimensions: 
Cognitive  Efficiency  and  Motivation  (see  table  2).  For  the  Intrinsic  Motivation  Inventory,  the  distinction 
between  the  "flow"  and  "no  flow"  conditions  was  found  in  the  two  subdimensions  of  Interest-Enjoyment 
and  Perceived  Competence  (see  table  3).  These  results  were  in  the  hypothesized  direction,  that  is,  the  means 
of  the  "flow"  group  were  greater  than  the  means  of  the  "no  flow"  group.  In  general,  there  is  evidence  to 
support  the  perceived  relationship  between  flow  and  measures  of  intrinsic  motivation,  as  predicted  by  the 
theory. 
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TABLE  2: 

MANOVA  :  ESF  subdimensions  by  Flow  -  No  Flow  condition 


Source  df  Type  III  SS  MS 

Affective 

FLOW/NO  FLOW  1  0.0002  0.0002  0.00 

Error  99  47.446  0.479 

Activation 

FLOW/NO  FLOW  1  1.101  1.101  2.37 

Error  99  45.946  0.464 

Cognitive  Efficiency 

FLOW/NO  FLOW  1  1.722  1.722  8.19  * 

Error  99  20.818  0.210 

Motivation 

FLOW/NO  FLOW  1  3.649  3.649  16.59 

Error  99  21.779  0.219 


p<  .05 


TABLE  3: 

MANOVA:  IMI  subdimensions  by  Flow  -  No  Flow  condition 


Source  df       Type  III  SS  MS  F 
Interest-Enjoyment 

FLOW/NO  FLOW  1         5.240  5.240  '  4.78* 

Error  104       39.699  0.992 

Perceived  Competence 

FLOW/NO  FLOW  1         13.314  13.314  10.35* 

Error  104        133.783  1.286 

Effort-Importance 

FLOW/NO  FLOW  1  2.502  2.502  2.38 

Error  104        109.519  1.053 

Tension-Pressure 

FLOW/NO  FLOW  1  3.013  3.013  2.00 

Error  104       202.211  1.944 

*  p<  .05 


The  results  of  this  analysis  generated  evidence  to  support  the  basic  assumption  of  flow  theory,  that 
is,  the  ratio  of  a  subject's  challenge  level  to  their  skill  level  can  be  used  as  an  indicator  of  intrinsic 
motivation.  It  should  be  noted  that  the  use  of  the  ESF  in  this  study  differs  in  certain  regards  from  the 
original  instrument  designers'  intention.  The  ESF  was  originally  used  in  time  series  studies  where  each 
subject  reported  their  responses  to  a  wide  variety  of  experiences  over  a  fixed  i>eriod  of  lime.  This  use  of  the 
instrument  was  in  a  controlled  setting  where  a  subject  reported  a  single  response  to  the  experimental 
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stimuli.  This  study  showed  that  the  ESF  instrument  can  be  useful  in  an  instructional  setting  and  is  capable 
of  distinguishing  between  the  "flov/'  and  "non-flow"  states  among  subjects. 

The  hypothesis  of  a  positive  relationship  between  measures  of  intrinsic  motivation  and 
achievement  was  generally  confirmed.  The  ESF  hid  a  small  to  moderate  overall  correlation  on  the  various 
measures  with  significant  correlations  in  the  Cognitive  Efficiency  and  Motivation  subdimensions.  The  IMI 
data  also  showed  moderate  overall  correlations  on  all  of  the  four  measures,  with  significant  correlations  in 
the  Interest-Enjoyment,  Perceived  Competence  and  Effort-Importance  subdimensions.  This  link  between 
intrinsic  motivation  and  achievement  has  often  been  assumed  in  the  literature,  but  until  now  little  empirical 
evidence  has  been  presented  (see  tables  4  and  5). 


TABLE  4: 

Correlation  between  ESF  subdimensions  and  Post  Test: 


Subdiminsion 


N 


Pearson  r 


Pr  rho  >  0** 


Overall: 
Affective 
Activation 
Cognitive  Efficiency 
Motivation 


100 
97 
91 
98 


0.072 
0.175 
0.438 
0.374 


0.47 
0.08 
0.0001  * 
0.0001  * 


*  Significant  at  the  0.05  level 
**  one-tailed  test 


TABLE  5: 

Correlation  between  IMI  subdimensions  and  Post  Test: 


Subdiminsion 


N 


Pearson  r 


Pr  rho  >  0** 


Overall: 

Interest-Enjoyment 
Perceived  Competence 
Effort-Importance 
Tension-Pressure 


91 
106 
106 
105 


0.381 
0.4470 
0.259 
0.112 


0.002  * 
0.0001  * 
0.007  * 
0.25 


*  Significant  at  the  0.05  ievel 
**  one-tailed  test 


The  third  part  of  this  study  was  exploratory  in  nature  and  was  concerned  with  discovering  the 
influence  of  prior  interest  and  prior  knowledge  on  intrinsic  motivation.  A  common  sense  assumption 
underlying  this  part  of  the  study  is  that  prior  interest  is  a  direct  measure  of  an  individual's  motivation 
toward  a  subject.  In  addition,  since  photography  is  rarely  taught  as  a  required  topic,  it  was  assumed  that  any 
prior  knowledge  of  this  subject  was  voluntary  on  the  part  of  the  individual,  and  hence  an  indirect  measure  of 
intrinsic  motivation. 

In  order  to  explore  the  relationship  between  prior  interest,  prior  learning,  and  intrinsic  motivation, 
analysis  of  variance  and  multiple  regression  analysis  were  used. 

An  ANOVA  procedure  was  designed  to  see  if  subjects  who  reported  a  high  level  of  prior  interest  and/or  a 
high  level  of  prior  knowledge  would  report  higher  levels  of  intrinsic  motivation.  Results  showed  that  high 
prior  interest  subjects  did  report  higher  scores  for  the  Affective  and  Cognitive  Efficiency  subdimensions  of 
the  ESF  (see  table  6),  and  high  prior  knowledge  also  resulted  in  higher  scores  for  the  Perceived  Competence 
subdimension  of  the  IMI. 
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TABLE  6: 

ESF:  ANOVAs  for  Exploratory  Data  Analysis 
(Treatment,  Prior  Interest  and  Prior  Knowledge) 


Type  I 

Source 

cjf 

S_£ 

MS 

E 

Affective  Subdimension 

Treatment 

2 

0.095 

0.047 

0.12 

Prior  Interest 

1 

2.161 

2.161  • 

5.24  * 

Treatment  X  Prior  Interest 

2 

0.000 

0.000 

0.00 

Prior  Knowledge 

1 

0.052 

0.052 

0.13 

Treatment  X  Prior  Knowledge 

2 

0.514 

0.257 

0.62 

Treatment  X  Prior  Interest 

X  Prior  Knowledge 

3 

3.41 1 

1. 137 

2.76  * 

Within  (error) 

88 

36.294 

0.412 

TOTAL 

99 

42.530 

Activation  Subdimension 

Treatment 

2 

0.761 

0.381 

0.89 

Prior  Interest 

1 

0.608 

0.608 

t    A  1 

1.41 

Treatment  X  Prior  Interest 

2 

0.991 

0.496 

1.15 

Prior  Knowledge 

1 

0.119 

0.119 

0.28 

Treatment  X  Prior  Knowledge 

2 

0.070 

0.035 

0.08 

Treatment  X  Prior  Interest 

X  Prior  Knowledge 

3 

1.314 

0.438 

1.02 

Within  (error) 

85 

36.560 

0.430 

TOTAL 

96 

40.424 

Cognitive  Efficiency  Subdimension 

Treatment 

2 

2.173 

1.087 

5.23  * 

Prior  Interest 

1 

1.361 

1.361 

6.55  * 

Treatment  X  Prior  Interest 

2 

0.141 

0.017 

0.10 

Prior  Knowledge 

1 

0.119 

0.071 

0.34 

Treatment  X  Prior  Knowledge 

2 

0.070 

0.035 

0.08 

Treatment  X  Prior  Interest 

X  Prior  Knowledge 

3 

0.625 

0.208 

1.00 

Within  (error) 

75 

15.587 

0.207 

TOTAL 

86 

20.432 

Motivation  Subdimension 

Treatment 

2 

0.296 

0.148 

0.72 

Prior  Interest 

1 

0.770 

0.770 

3.77 

Treatment  X  Prior  Interest 

2 

0.658 

0.329 

1.61 

Prior  Knowledge 

1 

0.012 

0.012 

0.06 

Treatment  X  Prior  Knowledge 

2 

0.028 

0.014 

0.54 

Treatment  X  Prior  Interest 

X  Prior  Knowledge 

3 

0.328 

0.109 

0.54 

Within  (error) 

86 

17.581 

0.204 

TOTAL 

97 

19.673 

*  p  <  .05  

In  addition,  the  generative  verses  supplantive  approach  to  learning  strategies  (Smith  &  Ragan, 
1993)  prescribes  that  individuals  with  high  prior  interest  and  high  prior  knowledge  would  benefit  from  a 
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more  generative  strategy,  whereas  low  interest/low  prior  knowledge  individuals  would  require  a  more 
supplantive  strategy.  Generative  strategies  are  more  motivating,  whereas  supplantive  strategies  tend  to  be 
less  challenging  and  less  motivating.  In  this  study,  treatment  #3  was  designed  to  be  more  generative  while 
treatment  #1  took  a  more  structured,  supplantive  approach.  This  ANOVA  was  able  to  test  the  predictions 
of  the  generative-supplantive  hypothesis  by  adding  a  third  dimension  and  testing  for  a  treatment  x  prior 
interest  x  prior  knowledge  interaction.  This  interaction  was  found  in  the  Affective  subdimension  of  the 
ESF.  A  post  hoc  analysis  revealed  that  the  direction  of  the  prediction  was  generally  confirmed,  with  the 
high  prior  interest/high  prior  knowledge  group  in  treatment  #3  reporting  the  highest  affective  scores. 
However,  the  low  prior  interest/low  prior  knowledge  group  also  reported  its  highest  mean  scores  for 
treatment  #3,  with  treatment  #1  having  the  second  highest  score. 

The  results  of  this  interaction  produced  evidence  supporting  the  generative-supplantive  hypothesis 
while  also  linking  it  to  a  theoretically  grounded  methodology  for  studying  intrinsic  motivation.  It  is 
notable  that  this  interaction  was  found  in  results  which  earlier  could  not  find  the  generalized  superiority  of 
one  strategy  over  another. 

In  the  stepwise  multiple  regression  analysis,  prior  interest  acted  as  a  significant  predictor  in  all 
four  subdimensions  of  the  ESF,  with  prior  knowledge  acting  as  a  significant  predictor  in  the  Activation  and 
Cognitive  Efficiency  subdimensions.  However,  the  total  variance  accounted  for  by  a  combination  of  these 
two  factors  was  never  greater  than  .10  (see  table  7).  Analysis  for  the  IMI  resulted  in  prior  knowledge  as  a 
significant  predictor  in  the  Perceived  Competence  and  Tension-Pressure  subdimensions;  however  only  a 
maximum  of  .06  of  the  variance  could  be  accounted  for  by  this  factor  (see  table  8).  Overall,  the  amount  of 
variance  accounted  for  by  the  predictor  variables  was  very  small. 


TABLE  7: 

ESF:  Multiple  Regression  for  Exploratory  Data  Analysis 
 (Prior  Interest  and  Prior  Knowledge)  

Stepwise 
Multiple 

Correlation  (R)        Stepwise  r2 
Source  Beta  r2  Increment 

Affective 

Prior  Interest  0.27  0.27  0.07 
Activation 

Prior  Interest  0.21  0.17  0.03 

Prior  Knowledge  0.21  0.27  0.07  0.04 

Cognitive  Efficiency 
Prior  Interest  0.27  0.23  0.05 

Prior  Knowledge  0.21  0.31  0.10  0.05 

Motivation 

Prior  Interest  0.24  0.06  0.06 


Note.  Table  includes  only  those  variables  that  met  the  .15  level  of  significance 
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TABLE  8: 


IMI:  Multiple  Regression  for  Exploratory  Data  Analysis 
 (Prior  Interest  and  Prior  Knowledge)  


Stepwise 
Multiple 
Correlation  (R) 


R2 


Source 


Beta 


Increment 


Perceived  Competence 
Prior  Knowledge 


0.25 


0.25 


0.06 


Tension-Pressure 
Prior  Knowledge 


0.19 


0.19 


0.04 


Note.  Table  includes  only  those  variables  that  met  the  .15  level  of  significance 

The  lack  of  variance  accounted  for  by  prior  interest  and  prior  knowledge  was  surprising  since  it  was 
anticipated  that  these  two  factors  would  have  a  strong  predictive  effect  on  intrinsic  motivation.  These 
results  may  be  due  to  measurement  error,  the  prior  interest  inventory  and  the  pre-test  were  both  researcher 
designed  instruments  and  have  not  beon  extensively  validated.  However,  it  is  doubtful  that  poor 
instrumentation  could  completely  account  for  the  extremely  low  variance  found  in  the  multiple  regression. 
It  is  likely  that  other,  unknown  variables  account  for  much  of  the  variance  in  the  intrinsic  motivation 
scores. 

Discussion  of  Findings 

The  results  of  this  study  suggest  that  attempting  to  increase  intrinsic  motivation  through  the 
design  of  instructional  materials  is  more  difficult  than  the  literature  would  indicate.  Investigating  the  data 
for  the  ESF  leads  to  some  general  conclusions  about  the  difficulties  of  integrating  intrinsic  motivational 
prescriptions  into  the  practice  of  instructional  design. 

It  was  surprising  that  the  Activation  subdimension  scores,  containing  questions  on  alertness, 
involvement,  excitement,  etc.  did  not  distinguish  between  the  treatments.  It  was  expected  that  treatment  #3 
would  have  a  stronger  impact  on  subjects  in  this  dimension  because  of  its  simulation  and  gaming 
strategies.  However,  mean  scores  for  all  three  treatments  on  these  questions  were  at  or  below  the  midpoint 
of  the  scale,  indicating  that  these  treatments  did  not  generate  a  very  active  sense  of  engagement  in  the  task. 
It  is  possible  that  the  design  of  treatment  #3  may  not  be  challenging  enough  for  this  particular  group  of 
subjects.  Defining  the  optimal  challenge  requires  a  careful  analysis  of  the  audience,  and  probably  varies 
widely  from  one  individual  to  another,  thereby  greatly  compounding  the  problem  for  the  designer. 

However,  the  Affect  subdimension  scores  were  above  the  midpoint  of  the  scale,  indicating  a 
positive  sense  of  happiness,  sociability,  and  relaxation  among  the  subjects.  These  questions  probably 
reflect  a  general  orientation  more  strongly  influenced  by  the  subject's  self  perception  or  world  view  than  a 
reaction  to  a  specific  activity. 

Cognitive  Efficiency  was  another  subdimension  where  theory  would  have  predicted  a  strong  effect 
for  treatment  #3.  This  subdimension  contained  questions  on  concentration,  expectation  and  satisfaction.  It 
was  expected  that  the  simulation  elements  of  treatment  #3,  which  provided  a  greater  amount  of  problem 
solving,  would  have  contributed  to  greater  levels  of  concentration  by  the  subjects.  In  addition,  the  gaming 
element  could  have  contributed  to  a  greater  sense  of  satisfaction.  However,  results  show  that  subjects  in 
treatment  #3  had  the  lowest  means  for  any  of  the  comparison  groups  on  this  subdimension,  although  this 
distinction  is  not  significant.  Again,  it  could  be  concluded  that  the  gaming  and  simulation  elements  of 
treatment  #3  were  not  challenging  or  extensive  enough  to  make  a  difference  among  the  subjects. 

The  motivation  subdimension  had  mixed  results.  For  questions  relating  to  the  subject's  sense  of 
control,  success  and  satisfaction  in  all  three  treatments,  respondents  scored  above  the  midpoint  of  the  scale. 
But  for  the  questions  relating  to  the  importance  of  the  activity  to  themselves  or  to  their  overall  goals. 
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respondents  scored  at  or  below  the  midpoint.  It  is  possible  that,  although  the  subjects  found  the  activity  to 
be  satisfying,  they  did  not  find  it  to  be  very  relevant  to  the  larger  context  of  their  overall  goals. 

The  only  significant  comparison  was  between  treatments  2  and  3  on  the  Perceived  Competence 
Subdimension  of  the  IMI.  The  comparison  of  this  subdimension,  which  measures  respondents'  sense  of 
competence,  skill  and  confidence  toward  the  activity,  showed  that  the  mean  scores  for  the  treatment  #2 
group  were  greater  than  the  mean  scores  for  treatment  #3,  counter  to  the  predicted  direction.  Since  the 
distinguishing  characteristics  of  treatment  #2,  the  non-linear  hypermedia  structure,  was  also  present  in 
treatment  #3,  it  is  possible  that  the  significance  of  this  comparison  is  a  statistical  fluke,  and  would 
disappear  in  subsequent  replications  of  the  study. 

This  study  points  out  the  complex,  interactive  nature  of  intrinsic  motivation.  If  it  is  assumed  that 
variables  affecting  any  type  of  psychological  phenomena  can  be  classified  into  environmental,  social,  and 
intrapersonal  factors,  then  it  can  be  seen  that  this  study  controlled  for  only  environmental  factors,  that  is, 
the  design  of  the  treatments.  Therefore  it  can  be  hypothesized  that  social  factors,  such  as  group  interaction, 
and/or  intrapersonal  factors,  such  as  cognitive  styles,  probably  have  a  stronger  impact  on  intrinsic 
motivation  than  environmental  factors  alone.  As  found  in  the  investigation  of  the  interaction  between 
treatments  and  the  factors  of  prior  interest  and  prior  knowledge,  it  is  likely  that  these  social  factors  will  be 
seen  to  operate  in  interaction  with  critical  environmental  factors  such  as  instructional  strategies. 

Conclusion 

This  study  develops  evidence  that  the  design  of  instruction  can  have  an  effect  on  intrinsic 
motivation,  and  that  intrinsic  motivation  has  an  effect  on  achievement,  and  that  methodologies  derived  from 
flow  theory  are  of  practical  use  in  an  instructional  setting.  However,  the  relationships  are  not  clear,  and 
there  appear  to  be  many  other  variables  affecting  learners'  intrinsic  motivation.  Further  research  should  be 
conducted  to  both  refine  the  motivational  appeal  of  instructional  materials  and  to  identify  those  social  and 
interpersonal  factors  that  impact  motivation. 
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Abstract 


Students'  academic  achievement  scores  have  been  found  to  improve  with  the  use  of  graphic  organizers. 
Researchers  suggest  this  may  be  due  to  the  way  graphic  organizers  depict  concepts  and  relationships 
between  concepts.  Unfortunately,  most  previous  research  on  graphic  organizers  has  been  constrained  to 
linglish  speakers  in  secondary  and  higher  education.  To  expand  our  knowledge  on  graphic  organizers  and 
increased  learning,  the  effecti  veness  of  graphic  organizers  when  used  by  fourth-grade  students  engaged  in 
computer-based  instruction  was  examined.  Additionally,  whether  the  learners1  dominant  language  (Spanish 
and  English)  influenced  the  effectiveness  of  graphic  organizers  was  studied.  A  one-way  analysis  of  variance 
was  used  to  analyze  immediate  and  delayed  gain  in  academic  achievement  scores  of  all  students.  Statistically 
and  educationally  significant  differences  were  found  in  scores  favoring  students  who  used  graphic  organizers 
over  those  who  used  lists  of  topics.  A  two-way  analysis  of  variance  identified  no  significant  differences 
between  language  groups  for  the  effect  of  graphic  organizers  on  immediate  or  delayed  tests.  Limitations  of 
the  study  and  suggestions  for  future  research  are  examined. 
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Background 


Introduction 

Cognitive  psychologists  generally  agree  that  there  are  at  least  two  distinct  types  of  human  knowledge  -- 
declarative  and  procedural  (Anderson,  1983;  Jonassen,  Beissner,  &  Yacci,  1993).  Declarative  knowledge  is 
knowing  what  something  is,  but  it  does  not  imply  knowing  how  to  use  that  knowledge.  Defining  the  word 
"computer"  and  identifying  the  number  24  on  a  number  line  are  examples  of  declarative  knowledge. 
Procedural  knowledge  is  knowing  how  to  do  something.  Knowing  how  to  manipulate  physical  entities  such 
as  accessing  a  computer  application,  or  mental  manipulations  such  as  subtracting  two-digit  numbers,  are 
examples  of  procedural  knowledge.  Procedural  knowledge  is  dependent  on  an  awareness  of  an  object  or  idea 
(declarative  knowledge)  even  though  .ve  may  not  be  able  to  articulate  that  knowledge  (Jonassen  et  al., 
1993). 

But  how  does  declarative,  knowledge  evolve  to  the  complex  forms  required  for  procedural  knowledge?  To 
help  explain  this  transition,  some  psychologists  propose  an  intermediate  knowledge  type  in  which  facts  and 
concepts  learned  as  declarative  knowledge  are  interrelated  with  one  another  to  form  complex  systems.  This 
interrelationship  of  information,  and  the  way  in  which  we  mentally  organize  it,  has  been  called  structural 
knowledge  (Diekhoff,  1983). 

Structural  knowledge  can  be  thought  of  as  a  network  of  mental  connections  or  relationships  between  pieces 
of  declarative  knowledge.  As  learners  develop  these  structures,  they  more  easily  associate  independent  ideas. 
These  connections  allow  learners  to  draw  conclusions  and  understand  relationships  among  concepts.  As 
students  begin  to  develop  these  underlying  structures  and  organizations  of  declarative  knowledge,  they  go 
beyond  "knowing  what"  to  a  better  understanding  of  "knowing  why."  Structural  knowledge  networks  appear 
to  create  the  interconnectedness  of  ideas  that  support  the  development  of  procedural  knowledge. 
A  variety  of  techniques  have  been  developed  to  elicit,  represent,  and  convey  structural  knowledge  (Jonassen 
et  al.,  1993).  One  method  being  employed  in  public  education  is  graphic  organizers.  Graphic  organizers  are 
spatial  metaphors  that  indicate  relationships  among  concepts  in  a  node-link-node  visual  display  (Anderson, 
1990;  Jonassen,  1990;  Jonassen  et  al.,  1993).  Nodes  contain  key  concepts.  Links  depict  unspecified 
relationships  between  nodes  (see  Figure  1). 
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Figure  1 .  Sample  graphic  organizer. 

Graphic  organizers  convey  relationships  and  content  structures  in  a -.  .torial  fashion.  These  visual 
representations  provide  lear.ers  with  a  structural  overview  of  information  to  be  learned.  This  overview 
directs  learner's  attention  towards  key  concepts  and  conceptual  relationships  rather  than  seemingly  isolated 
facts.  The  use  of  graphic  organizers  enhances  the  understanding,  organization,  and  long-term  retention  of 
information  (Stevensold  &  Wilson,  1990)  and  accentuates  meaningful  learning  and  information 
manipulation  (Jonassen,  1990;  Kerchner,  1990;  Peel,  1992).  Graphic  organizers  also  facilitate  the 
extrapolation,  combination,  inference,  and  other  logical  reasoning  mechanisms  that  allow  learners  to 
transfer  and  apply  information  (Jonassen  et  al.,  1993). 
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Problem  Statement 

Graphic  organizers  improve  the  effectiveness  of  instruction.  However,  it  cannot  be  assumed  that  they  affect 
all  students  equally.  In  fact,  researchers  suggest  that  the  effectiveness  of  any  instructional  strategy  varies 
with  individual  learner  differences  (Dunn,  1990;  Oxford  1990;  Scarpaci  &  Fradd,  1985). 
Individual  learner  differences  include  any  personal  trait  or  capability  that  influence  the  way  students  process 
and  use  information.  Although  there  are  a  variety  of  differences  that  affect  learning,  dominant  language  is  a 
key  difference  because  it  serves  as  a  central  means  for  information  processing  (Sticht,  1992).  Dominant 
language  differences  do  not  imply  a  difference  in  intelligence,  but  rather  in  the  way  individuals  organize  and 
use  information  (Scarpaci  &  Fradd,  1985). 

Previous  research  on  the  effectiveness  of  graphic  organizers  generally  focused  on  Anglo-Americans  in 
secondary  and  higher  education.  In  a  review  of  the  literature,  no  studies  were  identified  that  included  a 
bilingual  (Spanish  —  English)  elementary  school-aged  population  engaged  in  computer-based  instruction. 
This  omission  limits  the  generalizability  of  previous  research  findings  as  to  the  effectiveness  of  graphic 
organizers  to  various  populations. 

Purpose  of  Study 

The  purpose  of  this  study  was  two-fold:  (a)  to  assess  the  effectiveness  of  graphic  organizers  in  computer- 
based  instruction  for  fourth-grade  students,  and  (b)  to  determine  the  relationship  between  learner's  dominant 
language  (Spanish  or  English)  and  the  effectiveness  of  graphic  organizers  in  computer-based  instruction  by 
comparing  mean  gain  scores  across  language  groups.  To  accomplish  this  assessment,  the  experiment  was 
structured  so  that  gains  in  academic  achievement  scores  of  two  groups  of  fourth-grade  students  after  they 
engaged  in  computer-based  instruction  could  be  compared.  One  group  accessed  an  instructional  program 
with  embedded  graphic  organizers;  the  second  group  accessed  an  analogous  instructional  program,  but  with 
embedded  lists  of  topics.  The  term  embedded  refers  to  features  contained  within  the  programs.  Gain  score 
differences  between  pretest  and  posttests  (immediate  and  delayed)  were  used  to  determine  whether  the  type  of 
embedded  feature  resulted  in  a  difference  in  short-term  and  long-term  recall. 


Methods 

Population  and  Sample 

The  population  under  study  was  fourth-grade  students  engaged  in  computer-based  instruction.  Students 
enrolled  at  a  single  elementary  school  in  a  suburb  of  San  Diego,  California,  served  as  the  accessible 
population.  The  sample  used  in  this  study  was  68  fourth-grade  students.  Two  fourth-grade  classes  with 
different  teachers  were  included.  The  sample  included  31  English  speaking  and  37  Spanish  speaking 
students.  English  and  Spanish  speakers  were  defined  by  each  individual's  dominant  language  of  instruction 
according  to  school  authorities.  We  dropped  three  students  from  the  study:  Two  students  did  not  submit 
pretests  and  a  third  did  not  submit  the  delayed  posttest.  Each  subject  who  completed  all  components  of  this 
study  («  =  65),  participated  in  a  pretest,  instructional  program,  immediate  posttest  (Posttest  1),  and  delayed 
posttest  (Posttest  2). 

Two  important  variables  led  us  to  choose  the  location  for  our  study.  First,  58%  of  the  school's  students  are 
classified  as  Limited  English  Proficiency  (LEP)  or  Non  English  Language  Background  (NELB)  students.  As 
such,  this  district  represents  a  microcosm  of  demographic  changes  that  will  soon  be  facing  many  California 
and  United  States  schools  in  the  near  future  as  enrollment  of  sheltered  English  students  increases.  The  LEP 
and  NELB  subjects  in  this  school  are  almost  all  Spanish  speakers  with  Mexican  cultural  backgrounds. 
Second,  all  students  between  second-grade  and  fifth-grade  spend  approximately  20  minutes  per  day  engaged 
in  computer-based  instruction.  Because  of  this  exposure,  and  because  the  study  involved  computer-based 
instruction,  test  results  should  not  be  contaminated  by  the  novelty  effect  that  may  have  influenced  results  at 
other  research  sites. 

Materials 

Instructional  programs.  The  instructional  material  included  four  versions  of  a  computer-based 
program  developed  using  IBM's  "LinkWaylM"  software.  The  programs  were  designed  to  include  content 
normally  presented  in  the  school's  fourth-grade  science  curriculum  and  structured  so  that  students  could 
complete  the  instruction  in  20  to  30  minutes. 

We  developed  four  analogous  versions  of  the  instructional  program.  Two  versions  were  developed  in 
English  and  two  in  Spanish.  In  each  language,  we  developed  one  program  with  embedded  graphic  organizers 
and  one.  with  embedded  lists  of  topics.  The  difference  between  the  programs  was  the  position  of  concept 
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labels  on  the  screen  and  the  use  of  lines  connecting  these  concept  labels  in  the  graphic  organizers. 
Otherwise,  the  programs  were  identical.  Figure  2  shows  partial  renditions  of  screens  depicting  this 
difference. 
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Figure  2.  Renditions  of  lists-of-topics  and  graphic-organizer  screens. 

The  programs  consisted  of  21  screens  containing  instructional  information,  7  screens  representing  either 
lists  of  topics  or  graphic  organizers,  and  3  screens  containing  user  directions  and  general  information 
regarding  the  program's  presentation. 

Screens  that  presented  lists  of  topics  and  graphic  organizers  were  designed  with  three  colors.  Green 
identified  what  material  users  would  see  next  in  each  treatment.  Pink  identified  what  was  just  viewed,  and 
white  indicated  the  overall  content  of  material.  The  colors  used  in  the  list-of-topics  and  graphic-organizer 
screens  served  fcur  purposes: 

•  to  provide  feedback  to  the  learners  on  their  progress  through  the  program; 

•  to  serve  as  a  preview  strategy,  highlighting  material  to  be  learned  (green); 

•  to  serve  as  a  review  strategy,  highlighting  material  just  learned  (pink);  and 

•  to  serve  as  a  synthesized  overview  of  all  the  information  in  the  program  (white). 

Test  Instruments.  We  developed  two  versions  of  the  test  to  match  the  dominant  language  (English 
or  Spanish)  of  the  students.  The  pencil-and-paper  academic  achievement  test  consisted  of  1 8  multiple  choice 
questions  from  the  facts,  concepts,  and  rule/principle  levels  of  Gagn6's  (1977)  taxonomy  of  cognitive 
skills.  Teachers  at  the  treatment  school  conducted  a  face-validity  measure  of  the  test  instrument  prior  to  its 
use  with  students.  We  used  the  test  to  gather  pretest,  immediate  posttest,  and  delayed  posttest  scores. 
Treatment 

A  stratified  sample  of  dominant  English  and  dominant  Spanish  speakers  were  randomly  assigned  to  two 
treatment  groups:  Treatment  1  students  accessed  graphic-organizer  screens  in  the  instructional  program; 
Treatment  2  students  accessed  list-of-topics  screens  in  an  otherwise  identical  version  of  the  program.  Within 
both  treatment  groups,  students  accessed  instruction  developed  in  their  dominant  language.  Dominant 
language  (English  and  Spanish)  and  program  type  (graphic  organizer  and  list  of  topics)  served  as  independent 
variables.  Mean  gain  score  differences  of  achievement  tests  served  as  the  dependent  variables  for  group 
results. 

The  instructional  program  was  limited  to  a  one-shot,  20  minute  treatment.  All  students  were  provided  with 
a  later  opportunity  to  use  the  non-treatment  program  to  ensure  equal  educational  opportunities  to  all 
subjects. 
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The  teachers'  role  was  limited  to  administering  the  pretests,  immediate  posttest,  and  delayed  posttest,  and 
directing  students  to  the  computer  where  they  accessed  their  pre-assigned  instructional  program. 
The  administration  procedures  for  this  study  involved  four  steps.  First,  all  students  took  the  academic 
precest  matched  to  each  individual's  dominant  language.  Approximately  one  day  later,  students  began  using 
the  treatment  program.  Because  each  classroom  had  only  one  available  computer,  students  took  turns  using 
the  program.  After  completing  the  program,  students  took  the  immediate  posttest  to  measure  their  short- 
term  recall.  Approximately  two  weeks  after  the  treatment  students  took  delayed  posttest  to  measure  their 
long-term  recall. 

Experimental  Design 

An  analysis  of  variance  (ANOVA)  statistical  procedure  was  used  to  identify  whether  statistically  significant 
differences  existed  between  group  mean  gain  scores.  Gain  scores  were  defined  as  the  difference  between 
pretest  and  immediate  posttest  scores  (Gain  1),  and  the  difference  between  pretest  and  delayed  posttest  scores 
(Gain  2).  Gain  scores  were  used  to  avoid  the  impact  of  individual  differences  on  the  level  of  content 
knowledge  at  the  start  of  the  study.  Dominant  language  and  program  type  were  used  as  independent 
variables.  Gain  1  and  Gain  2  scores  served  as  dependent  variables. 

We  used  a  one-way  ANOVA  to  determine  if  statistically  significant  differences  existed  between  the  means  of 
the  two  treatment  groups.  To  identify  if  statistically  significance  differences  existed  among  the  four 
subgroups  (two  languages  by  two  treatments)  a  two-way  ANOVA  was  conducted.  A  predetermined  overall 
level  of  statistical  significance  was  set  at  0.05.  The  predetermined  level  of  educational  significance,  as 
measured  by  the  standardized  mean  difference  effect  size  (calculated  by  dividing  mean  differences  of  the 
posttest  by  the  standard  deviations  of  the  scores  from  the  untreated  groups)  was  set  at  0.25  (Tallmadge, 
1977). 

Results 

Overview  of  Descriptive  Statistics 

Table  1  presents  mean  gain  scores,  standard  deviations,  and  sample  sizes  for  the  four  treatment  groups 
involved  in  this  study.  Mean  gain  scores  are  presented  for  two  instances:  Gain  1  (defined  as  the  difference 
between  immediate  posttest  scores  and  pretest  scores)  and  Gain  2  (defined  as  the  difference  between  delayed 
posttest  scores  and  pretest  scores). 


Table  1 

Means,  Standard  Deviations,  and  Sample  Sizes  for  All  Groups 


Treatment 

Gain  1   Gain  2 
Groups 

M 

SD 

M 

SD 

n 

List  of  Topics 

English 

2.61 

1.56 

1.01 

1.04 

13 

List  of  Topics 

Spanish 

3.58 

2.17 

1.63 

1.85 

19 

Subtotal 

3.19 

1.94 

1.14 

1.70 

32 

Graphic  Organizer 

English 

4.20 

2.27 

2.13 

1.64 

15 

Graphic  Organizer 

Spanish 

4.17 

1.95 

2.61 

2.03 

18 

Subtotal 

4.18 

2.07 

2.39 

1.71 

33 

Total 

3.69 

2.02 

1.90 

0.20 

65 

Subtotal 

Spanish 

3.87 

2.06 

2.12 

1.94 

37 

Subtotal 

English 

3.42 

1.91 

1.62 

1.30 

28 

Information  in  Table  1  reveals  that  Gain  2  scores  were  lower  than  Gain  1  scores  for  all  treatment  groups. 
The  reduction  in  gain  scores  reflects  an  expected  decrease  in  the  amount  of  information  retained  over  time. 
Initial  analysis  indicated  that  in  immediate  and  delayed  recall  tests,  mean  gain  scores  of  both  English  and 
Spanish  speaking  groups  using  programs  with  embedded  graphic  organizers  were  higher  than  those  achieved 
by  their  counterparts  using  programs  with  embedded  lists  of  topics.  These  results  lead  us  to  believe,  at  least 
with  descriptive  statistics,  that  embedding  graphic  organizers  in  instructional  programs  is  a  better  strategy 
to  improve  academic  gains  than  embedding  lists  of  topics  in  instructional  programs  for  both  Spanish  and 
English  speaking  fourth-grade  students. 


Statistical  and  Educational  Significance 

Graphic  organizers  versus  list  of  topics,  From  data  in  Table  1,  the  standardized  mean  difference 
effect  size  between  groups  was  calculated  using  computer-based  instructional  programs  with  embedded 
graphic  organizers  and  embedded  list  of  topics.  Effect  sizes  were  identified  to  be  0.40  for  Gain  1  scores  and 
0.5 1  for  Gain  2  scores.  An  ANOVA  was  conducted  for  both  Gain  1  and  Gain  2  scores  (see  Table  2). 


Table  2 

Analysis  of  Variance:  Comparison  of  Gain  I  and  Gain  2  for  English  and  Spanish  Speakers  Grouped  by 
Program  Type 


Dependent 

Source 

Degrees  of 

Sum  of 

Mean 

Ftest 

Significance  of 

Variable 

Freedom 

Squares 

Square 

one-tail  F  test 

Gain  1 

Between  groups 

1 

16.06 

16.06 

3.99 

p  =  .04 

Within  groups 

63 

253.78 

4.03 

Total 

64 

269.84 

Gain  2 

Between  groups 

1 

15.84 

15.85 

5.32 

p=  .01 

Within  groups 

63 

187.59 

2.98 

Total 

64 

203.44 

The  information  in  Table  2  indicates  that  there  was  a  statistically  significant  difference  between  treatment 
groups  favoring  graphic  organizers  on  both  short-term  (Gain  1  scores)  and  long-term  recall  (Gain  2  scores). 
Previous  researchers,  as  discussed  earlier,  indicated  that  graphic  organizers  within  instructional  material 
improved  short-term  and  long-term  information  recall.  Our  findings  support  these  conclusions.  This 
experiment  also  extended  research  into  the  realm  of  computer-based  instruction  and  elementary  students. 
Evidence  now  exists  which  supports  the  idea  that  the  use  of  graphic  organizers  in  computer-based 
instructional  programs  has  a  significant  positive  effect  on  both  short-term  and  long-term  recall  for  fourth- 
grade  students. 

Dominant  language  differences.  A  two-way  ANOVA  was  used  to  determine  if  the  effectiveness  of 
graphic  organizers  in  computer-based  instructional  programs  varied  significantly  by  learner's  dominant 
language.  In  this  computation,  program  type  and  dominant  language  were  used  as  independent  variables  and 
gain  scores  as  the  dependent  variable.  This  analysis  is  shown  in  Table  3. 


Table  3 

Analysis  of  Variance:  Comparison  of  Mean  Scores  for  Graphic  Organizer  Users  Grouped  by  Dominant 
Language 


Dependent  Source 

Degrees  of 

Sum  of 

Mean 

Ftest 

Significance  of 

Variable 

Freedom 

Squares 

Square 

one-tail  F  test 

Gain  1  Program 

1 

3.44 

3.44 

0.85 

p  =  .03 

Language 

1 

18.74 

18.74 

4.64 

p  =  36 

Program  X  Language 

1 

3.95 

3.95 

0.97 

p  =  33 

Error 

61 

246.61 

4.04 

Gain  2  Program 

1 

16.46 

16.46 

5.48 

p  =  .02 

Language 

1 

4.23 

4.23 

1.41 

p  =  .24 

Program  X  Language 

1 

0.02 

0.02 

0.01 

p  =  .93 

Error 

61 

183.36 

Information  presented  in  Table  3  indicates  that  there  is  no  statistically  significant  difference  at  the  0.05 
confidence  level  between  language  groups  for  the  effect  of  graphic  organizers  in  immediate  (Gain  1)  or 
delayed  tests  (Gain  2).  These  results  indicate  that  the  effectiveness  of  graphic  organizers  in  computer-based 
instructional  programs  does  not  vary  significantly  by  learner's  dominant  language.  The  standardized  mean 
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difference  effect  size  calculated  from  Table  1  was  0.01  for  Gain  1  scores  and  0. 19  for  Gain  2  scores.  These 
scores  indicate  that  differences  on  the  effect  of  graphic  organizers  between  language  groups  are  not 
educationally  significant  for  either  short-term  or  long-term  recall. 

Discussion 

Implications  of  the  Study 

Two  main  implications  can  be  deducted  from  the  results  of  this  study.  First,  the  use  of  programs 
with  embedded  graphic  organizers  in  computer-based  instruction  enhances  short-term  and  long-term  recall  in 
fourth-grade  students.  The  increase  in  academic  achievement  scores  measuring  long-term  and  short-term 
recall  confirmed  previous  research  findings.  Second,  results  indicate  that  dominant  language  differences  do 
not  significantly  impact  the  effectiveness  of  programs  with  embedded  graphic  organizers.  The  non- 
significant differences  between  dominant  Spanish  and  dominant  English  speaking  students  were  unexpected. 

Researchers  agree  that  the  effectiveness  of  any  learning  strategy  depends  on  context  and  content  of 
instruction,  and  on  individual  learner's  experience,  cognitive  maturity,  motivation,  and  learning  style 
(Oxford,  1990;  Schmeck,  1983).  In  this  study,  context  and  content  of  instruction  were  analogous  for  all 
treatment  groups.  One  assumption  made  was  that  subjects  did  not  substantially  differ  in  terms  of  their 
experience  and  cognitive  maturity  because  they  were  at  the  same  grade  level,  they  were  randomly  assigned  to 
treatment  groups,  and  their  pretest  scores  did  not  indicate  statistically  significant  differences  at  the  0.05  level 
(Gimenez,  1994).  Therefore,  our  results  indicated  that  when  using  computer-based  instructional  programs 
with  embedded  graphic  organizers,  Spanish  speaking  and  English  speaking  students,  as  groups,  did  not  vary 
on  leaning  style  to  the  point  of  significantly  impacting  academic  achievement  scores  according  to  tests  used 
in  this  study  to  measure  short-term  and  long-term  recall. 

Theoreticians  suggest  that  cultural  background  provides  the  most  important  resources  for  individual 
cognitive  development  (Connolly  &  Tucker,  1982;  Stein,  1990;  Sticht,  1992).  Such  resources  refer  to 
symbols  and  symbol  systems,  including  natural  language  and  conceptual  knowledge.  These  are  the  primary 
tools  for  the  transmission  of  cognitive  abilities  and  motivational  conditions  (Sticht,  1992).  Results  of  this 
study  indicated  that  the  different  cultural  backgrounds  of  dominant  Spanish  and  dominant  English  speaking 
subjects  did  not  significantly  influence  the  effectiveness  of  graphic  organizers  embedded  in  computer-based 
instructional  programs. 

In  searching  for  causes  of  academic  achievement  differences  between  dominant  English  and 
dominant  Spanish  speaking  students,  researchers  determined  differences  in  learning  style  preferences  as  key 
elements  (Scarpaci  &  Fradd,  1985;  Schaiper  &  Flores,  1985).  These  researchers  suggested  that  culture 
strongly  influences  the  way  in  which  students  learn  and  interact.  Instead  of  faulting  individual 
characteristics,  studies  on  LEP  and  NELB  native  Spanish  students  identified  that  low  scholastic  achievement 
was  related  to  instructional  materials,  instructional  methods,  classroom  environments,  and  learning 
strategies  that  favor  native  English  students  (Dunn,  1990;  Ulibarri,  1982).  Recent  studies  determined  that 
the  United  States  educational  system,  at  all  levels  of  instruction,  relies  on  the  learning  preferences  of  white 
middle-class  students,  and  these  preferences  are  not  always  shared  by  Mexican-Americans  (Cohen,  1969; 
Connolly  &  Tucker,  1982).  However,  results  of  this  study  indicated  that  graphic  organizers  embedded  in 
computer-based  instructional  programs  significantly  increased  mean  academic  achievement  scores  of  both 
groups,  and  did  not  significantly  vary  between  dominant  English  and  dominant  Spanish  speaking  students. 

Results  of  this  study  support  previous  theories  suggesting  that  the  use  of  graphic  organizers  is  an 
effective  strategy  to  increase  short-term  and  long-term  information  recall.  Although  both  groups  had  a 
decrease  in  Gain  2  scores  reflecting  long-term  recall,  this  decrease  was  more  pronounced  for  the  group  using 
programs  with  embedded  list  of  topics.  These  results  suggest  that  the  use  of  computer-based  instructional 
programs  with  embedded  graphic  organizers  have  their  greatest  impact  on  long-term  recall.  Although  the 
study  was  limited  to  a  one-shot  20  minute  treatment,  the  results  were  significant.  These  findings,  concluded 
from  a  single,  short  exposure  to  graphic  organizers,  foretell  of  potentially  major  cognitive  growth  in 
students  who  use  this  strategy  for  extended  p   iods  of  time.  Thus,  educators  and  instructional  designers  can 
strongly  encourage  the  use  of  embedded  graphic  organizers  in  instructional  materials  as  means  to  enhance 
short-term  and  long-term  recall  of  information. 

Limitations  of  the  Study 

Given  the  characteristics  of  the  study,  several  shortcomings  are  apparent.  First,  teachers  did  not  monitor 
students  during  the  treatment.  It  is  possible  that  students  did  not  complete  the  treatment  or  may  have  used 
the  program  assigned  to  a  different  group.  However,  data  were  consistent  with  previous  studies.  This 
consistency  lends  support  to  our  conclusions.  Second,  the  treatment  was  limited  to  a  one-shot,  20-minute, 
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computer-based  instructional  program.  Although  this  short  exposure  was  initially  considered  a  weakness  in 
our  study  design,  it  may  also  be  viewed  as  supporting  the  strength  of  graphic  organizers.  Because  students 
exhibited  significant  short-  and  long-term  gains  after  a  brief  exposure,  longer  exposures  to  graphic 
organizers  may  generate  even  greater  gains  in  academic  scores. 

Questions  for  Future  Research 

Questions  and  concerns  for  future  research  include  the  following: 

1.  Do  graphic  organizers  embedded  in  computer-based  instructional  programs  improve  academic 
achievement  scores  when  the  graphics  are  used  as  navigation  controls?  This  study  focused  on  the  use  of 
graphic  organizers  as  graphic  representations  of  information.  However,  literature  reveals  that  graphic 
organizers  in  computer-based  instruction  may  also  serve  as  navigational  tools  (Barba,  1993).  The  literature 
on  learner  control,  however,  is  controversial.  Some  researchers  suggest  that  because  graphic  organizers  may 
be  used  as  a  nonlinear  presentation  controller,  the  impact  on  the  instructional  process  may  vary  according  to 
the  learner's  cognitive  maturity,  previous  knowledge,  learning  style,  and  preferred  leaning  strategies. 
Researchers  determined  that  although  learner  control  is  motivating,  the  lack  of  structure  in  the  instructional 
sequence  may  cause  barriers  for  some  learners  (Malone  &  Lepper,  1987). 

2.  Does  the  use  of  graphic  organizers  in  computer-based  instruction  improve  academic  achievement 
scores  when  the  organizers  do  not  provide  feedback  on  learner  progress?  Although  this  study  focused  on  the 
use  of  graphic  organizers  as  graphic  representations  of  information,  these  organizers  also  served  to  provide 
feedback  on  learner  progress.  Feedback  was  provided  by  using  colors  to  identify  which  concepts  had  just 
been  explained,  which  were  about  to  be  viewed,  and  the  overall  structure  of  the  lesson.  Future  research  may 
replicate  this  study  without  the  use  of  feedback  to  determine  whether  this  feature  was  a  critical  variable  for 
the  effectiveness  of  graphic  organizers  in  this  study. 

3.  Does  the  use  of  graphic  organizers  in  computer-based  instruction  improve  academic  achievement 
scores  when  they  include  labeled  links?  Explicitly  labeling  the  relationships  in  a  graphic  organizer  may 
cause  information  overload  in  students  with  lower  cognitive  capabilities.  In  this  study,  labeled  relationships 
were  avoided  because  it  was  assumed  that  they  may  impose  extra  cognitive  processing  leading  to 
information  overload  for  fourth-grade  students.  Researchers  may  want  to  compare  the  effectiveness  of 
graphic  organizers  with  and  without  labeled  relationships  with  the  population  used  in  this  study. 

4.  What  are  the  characteristics  of  students  who  do  not  improve  academic  scores  when  they  use 
computer-based  instructional  programs  with  embedded  graphic  organizers?  A  further  study  could  identify 
characteristics  of  students  who  prefer  to  use  lists  of  topics  to  graphic  organizers  by  exposing  learners  to 
both  strategies  and  allowing  them  to  choose  the  strategy  that  lias  the  greatest  impact  on  their  short-term  and 
long-term  recall  (measured  in  terms  of  academic  achievement  scores).  By  identifying  attributes  of  both 
groups  of  students,  instruction  could  be  tailored  to  better  meet  individual  needs. 

5.  This  study  focused  on  Spanish  and  English  speakers  who  are  in  a  bilingual  environment. 
Researchers  may  wish  to  replicate  this  study  to  include  subjects  in  monolingual  educational  settings,  and  in 
settings  with  other  than  Spanish  and  English  speakers. 
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Ethnography 

An  ethnography  can  be  defined  as  an  in-depth  analytical  description  of  an  intact  cultural  scene. 
This  research  observes  human  behavior  in  its  natural  setting  (holistic  inquiry  carried  out  in  a  natural  setting) 
over  a  period  of  time.  Naturalistic  perspective  emphasize  humans  as  the  primary  data  gathering  instrument 
and  the  use  of  triangulated  data-gathering  procedures  (Goetz  &  LeCompte,  1984).  Human  observation  is 
flexible  and  can  adapt  to  a  complex  situation  as  it  evolves.  In  addition,  human  observation  allows  for  the 
identification  of  biases  that  result  from  interactions  and  value  differences  between  the  "instrument"  and  the 
subject  (Borg,  1989).  To  help  reduce  biases,  researchers 

should  use  triangulation  of  data.  Triangulation  is  the  process  of  collecting  and  examining  related  documents 
to  serve  as  crossreferences  to  observed  data.  Researchers  using  naturalistic  inquiry  purposely  select  their 
subject/group  or  setting.  By  purposeful  selection,  a  qualitative  researcher  will  be  more  likely  to  uncover 
the  full  array  of  "multiple  realities"  relevant  to  an  inquiry.  The  focus  of  the  ethnographer's  inquiry  is  on  the 
mundane,  everyday  practices  of  people.  The  paramount  objective  of  qualitative  research  is  understanding 
rather  than  generalizing  or  identifying  cause  and  effect  (Whitt,  1991). 

Pilot  Study 
Research  Design 

This  study  was  undertaken  to  explore  the  leadership  qualities  of  a  district  media  director.  The  school 
district,  Byrdsville  Public  School  District  or  BPS  as  school  personnel  referred  to  it  (This  is  a  pseudonym 
for  the  actual  school  district  for  confidentiality.),  is  a  mid-sized  school  district  with  26  school  buildings  and 
approximately  1 3,000  students.  Using  ethnographic  research  methods,  the  following  questions  were 
investigated: 

What  are  the  beliefs  and  values  which  underlies  this  media  director's  decisions? 

What  are  the  recurring  events  and  practices  of  the  media  director? 

What  are  the  relationships  that  media  director  must  cultivate? 

What  are  the  day-to-day  encounters  of  a  media  director? 

What  are  the  leadership  qualities  of  this  media  director? 
The  significance  of  this  study  revolves  around  the  lack  of  any  research  on  media  directors.  Research  has 
been  typically  focused  on  either  administrators  (superintendents  or  principals)  or  teachers. 

Data  for  this  study  were  obtained  through  observation,  interviews,  and  artifact  analysis. 
Respondents  were  guaranteed  confidentiality  and  all  data  presented  in  this  study  are  designed  to  protect  their 
anonymity. 

Roles  and  Experiences  of  Investigator 

During  this  study,  I  became  a  participant  observer,  an  intern  to  the  media  director.  My  internship 
was  contracted  as  one  day  a  week,  Tuesday,  for  two  semesters  so  I  could  observe  a  full  school  year  cycle.  In 
addition.  I  attended  special  events  I  was  invited  to.  By  being  actively  involved  in  the  ituation  being 
observed,  insights  and  interpersonal  relationships  developed  which  would  have  been  virtually  impossible  to 
achieve  through  any  other  method.  This  role,  participant  observer,  requires  the  researcher  to  come  to  the 
situation  as  a  learner  and  to  try,  at  least  partially,  to  be  socialized  into  the  group.  The  socialization  process 
allows  the  researcher  to  experience  the  physical,  relational,  and  emotional  realities  of  the  setting  as  it  exists 
for  the  members  (Dobbert,  1984). 

In  the  beginning  as  a  participant  observer,  Dobbert  (1984)  recommends  the  researcher  keep  in  the 
background,  keep  quiet,  watch,  and  listen.  In  addition,  the  researcher  should  be  maintaining  a  friendly  and 
willing  stance,  offering  to  assist  with  single  or  onerous  tasks,  accepting  invitations  and  assistance 
gratefully,  asking  questions  when  you  don't  think  it  would  be  bothersome,  and  trying  to  learn  the  language 
of  the  group  (Dobbert,  1984). 

Wolcott  ( 1988)  warns  that  taking  a  more  active  role  than  the  "observer"  has  costs  attached; 
research  efforts  are  sometimes  secondary  to  the  participation.  However,  the  benefits  of  being  a  participant- 
observer  includes:  I)  quality  of  direct  on-site  observation;  with  the  participant,  long  term  observation  and 
group  acceptance  is  more  likely  to  occur;  2)  freedom  of  access;  more  documents  and  subjects  are  available  to 
the  researcher  when  she  or  he  is  a  "part"  of  the  group;  3)  intensity  of  observation;  by  being  in  the  "loop" 
there  are  typically  more  hours  of  observation,  increasing  the  chances  of  observing  the  common  and  the 
exceptional;  and  4)  data  sampling;  by  being  a  participant,  the  researcher  may  be  included  in  meetings 
(formal  and  informal)  and  be  included  in  mailings  that  she  or  he  would  not  have  known  about  otherwise 
(Borg,  1989). 


Data  Collection  Methods 

"Ethnographers  consider  data  to  be  potentially  verifiable  information  obtained  from  the 
environment.  The  problem  of  data  collection,  then,  consists  of  defining  appropriate  information  and 
developing  strategies  for  obtaining  it"  (Goetz  &  LeCompte,  1984,  p.  107).  The  most  common  data 
collection  includes  observation,  interviewing,  researcher-designed  instruments,  and  content  analysis  of 
human  artifacts  (Goetz  &  LeCompte,  1984). 

Interactions  and  observations  were  the  primary  sources  for  the  field  notes  in  this  study.  Within 
these  activities  are  discussions  relative  to  the  media  director's  position.  With  each  observation  I  became 
more  at  ease  discriminating  between  what  should  and  should  not  be  recorded,  knowing  what  I  left  out  shaped 
my  data  as  certainly  as  that  which  I  included.  During  observations  I  would  jot  down  notes  and  quotes. 
Later,  these  would  be  rewritten,  expanding  my  description  of  the  experience. 

In  addition  to  the  field  notes,  informal  interviews  were  conducted  with  the  media  director, 
secretaries,  media  specialists,  and  principals.  I  used  interviewing  methods  to  "get  at"  the  reasons  for  certain 
actions  I  had  observed  or  "get  to"  the  individual's  opinion,  background,  and  experiences.  These  interviews 
ensured  a  variety  of  perspectives  and  consistency  of  data. 

Correspondence  or  documents  relating  to  the  media  director  were  collected  in  efforts  to  cross- 
reference  to  the  observed  data.  These  artifacts  were  typical  correspondence  of  the  media  director,  letters  and 
memos  that  were  sent  within  and  outside  of  the  district.   In  addition,  I  collected  original  material  Pete 
created  for  specific  groups  (media  specialists,  secretaries,  district  coordinators,  district  technology  group). 
This  process  allowed  for  triangulation  and  a  "trustworthiness"  of  the  data  (Lincoln  &  Guba,  1985). 

Respondents 

Following  Lincoln  &  Guba's  (1985)  suggestion,  purposeful  sampling  was  employed  in  order  to 
obtain  as  much  information  as  possible  and  to  get  the  maximum  variation  in  sampling.  Further,  the 
intended  purpose  of  data  collection  was  not  to  generate  enough  data  to  be  able  to  draw  conclusions  or 
generalizations  but  to  be  able  to  provide  results  with  the  uniqueness  of  the  subject  or  phenomenon  (Lincoln 
&Guba,  1985). 

All  respondents  in  this  pilot  study  were  employed  by  Byrdsville  Public  School  District.  Every 
respondent  was  told  they  would  have  anonymity  and  all  information  would  be  confidential.  Furthermore, 
each  respondent  was  given  a  pseudonym  either  by  choosing  it  herself/himself  or  one  was  provided  by  the 
researcher  in  order  to  maintain  confidentiality.  Respondents  who  volunteered  to  be  observed  and  to 
interviews  included: 

Media  director:  Dr.  Pete  Young 

Microcomputer  coordinator:  Bob 

Media  specialists:  Sharon,  Laura,  Brenda 

Principal:  Paul 

Secretaries:  Nancy,  Elizabeth,  Frances,  Susan 
In  addition,  there  were  many  individuals  who  were  a  part  of  the  project  because  of  their  interactions  with 
Pete.  I  was  introduced  as  "his  intern  who  is  learning  how  a  media  program  works." 

Analysis 

Data  does  not  answer  research  questions;  it  must  be  analyzed  before  a  scientific  perspective  can  be 
offered.  Lincoln  &  Guba  (1985)  promote  the  induction  and  constant  comparison  method  of  data  analysis. 
Dobbert's  (1984)  description  of  Darwinian  natural  history  method  follows  this  same  research  strategy.  The 
method  includes  five  steps  and  is  a  spiral  process.  The  five  steps  are.  1 )  careful  observation,  2)  have  a 
conceptual  scheme  into  which  to  try  and  fit  the  data  (this  occurs  with  step  1 ),  3)  change  the  concepts, 
assumptions,  or  hypotheses  to  accommodate  the  facts  (this  occurs  after  step  1 ),  4)  group  facts  by  categories 
and  arrive  at  conclusions  by  explaining  the  relationships  or  processes  that  create  the  differences  and 
similarities  expressed  in  the  categories  (this  occurs  with  step  3),  and  5)  turn  to  a  new  observation  directed 
by  the  discoveries  stemming  from  the  analytic  process  as  carried  out  through  the  previous  four  steps 
(Dobbert,  1984). 

Data  were  analyzed  and  categorized  into  issues  which  describe  the  inter-relatedness  of  theory  and  the 
decision  making  /  actions  of  the  media  director.  The  themes  of  the  study  are  presented  in  the  following  two 
sections:  1 )  Leadership:  Lead,  Follow,  or  Please  Retire!,  2)  Communication:  The  Way  I  Hear  It. 

Literature  Review:  Leadership 

Leadership  is  a  process  whereby  one  person  exerts  social  influence  over  the  members  of  the  group. 
A  leader  is  a  person  with  power  over  others  who  exercises  this  power  for  the  purpose  of  influencing  their 
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behavior  (Heller,  Van  Till  &  Zurcher,  1986).  Today,  leaders  are  challenged  as  they  find  themselves  working 
with  more  individuals  who  are  different  from  themselves  and  they  must  bridge  the  empathy  gap.  A  leader's 
creativity  must  be  used  when  working  with  the  resources  of  their  diverse  team  members  and  avoid 
neglecting  the  team  members'  important  skills  and  competencies. 

Educational  technology  leadership  shares  much  with  the  research  on  leadership.  However,  the 
specific  knowledge  and  abilities  connected  with  technology  provides  for  unique  considerations.  An  analysis 
of  the  effectiveness  of  technology  in  education  suggests  that  the  manner  in  which  technology  is 
implemented  is  more  important  than  any  intrinsic  characteristics  of  the  technology  (Kearsley  &  Lynch, 
1 992).  Leadership  plays  a  very  critical  role  in  the  process  of  the  adoption  i  f  technology. 

Technology  leaders  today  are  expected  to  possess  strong  visionary  skills,  have  excellent 
interpersonal  and  communication  skills,  as  well  as  a  measure  of  technical  competence  in  addition  to  the 
"basic  leadership"  capabilities  (Ray,  1992).  To  be  successful,  these  leaders  must  be  successful  in  their 
abilities  to  articulate  and  influence  cultural  norms  and  values;  they  are  expected  to  shape  the  culture  of  the 
school  by  creating  a  new  vision  that  members  can  believe  in  and  act  upon  (Kearsley  &  Lynch,  1992). 

Electronic  technology  has  been  viewed  as  a  radical  departure  from  the  traditional  educational 
technologies  (books,  chalkboards,  duplicating  machines,  teaching  systems,  ect.).  The  new  technology  has 
only  been  widely  available  or  affordable  since  the  mid  1980's.  Therefore,  the  majority  of  teachers  did  not 
have  the  opportunity  to  practice  with  electronic  technology  during  their  preprofessional  training.  Teachers 
are  typically  insecure  and  unskilled  in  regards  to  using  technology  in  their  classroom  and  mandates  to  use 
technology  by  administrators  only  leave  them  frustrated  and  floundering 

The  failure  of  any  technology  can  result  for  a  variety  of  reasons.  Some  of  these  reasons  can 
include  lack  of  teacher  training,  lack  of  appropriate  materials/equipment,  poorly  conceived  implementation 
plans,  unrealistic  goals/expectations,  or  insufficient  funding.  Most  of  these  pitfalls  can  be  corrected  with 
good  technological  leadership.  Technology  leaders  must  identify  educational  problems  which  technology 
can  solve  and  build  theoretical,  political,  and  financial  support  structures  tc  ensure  success  (Kearsley  & 
Lynch,  1992). 

Lead,  Follow,  or  Please  Retire 

The  first  time  I  was  in  Pete's  office,  was  to  discuss  my  internship  with  him.  Pete's  office  is 
located  in  an  old  brick  school  building  called  Smith  by  the  district.  Smith  is  located  in  the  center  of  the 
city  and  is  surrounded  by  federal  assistance  housing  projects.  The  east  side  of  Smith  is  its  architectural 
front  and  has  a  large  green  yard  between  it  and  a  major  street.  The  drive  entrance  is  located  behind  the 
building  with  a  small  paved  parking  lot  at  the  south  end  of  the  building  and  a  six  foot  fence  on  the 
perimeter  of  the  property.  The  doors  coming  into  the  building  from  the  parking  lot  are  the  doors  used 
primarily  by  the  Instructional  Materials  Services  (IMS)  department.  The  other  half  of  the  building  is 
occupied  by  the  alternative  high  school.  A  high  school  for  students  who  could  or  would  not  "fit"  into  the 
district's  traditional  high  schools. 

Upon  entering  from  the  parking  lot  there  is  a  hundred  yard  walk  down  a  dimly  light  hallway  filled 
with  furniture  and/or  equipment,  depending  of  the  current  activities  of  IMS,  to  Pete's  office.  His  office 
reflects  its  age  by  the  height  of  the  ceiling,  the  light  fixtures,  exposed  plumbing,  and  dilapidated  walls.  The 
rectangular  office  is  painted  yellow.  Yellow  is  considered  a  positive,  warm  color;  it  is  also  the  color 
disliked  by  the  most  people  (Raines,  1989).  With  his  desk  facing  the  entrance  of  the  office,  there  are  four, 
six  foot  high  windows  behind  him  that  create  a  halo  effect  with  the  afternoon  sunshine.  A  round  table  and 
four  chairs  occupy  the  front  of  the  office. 

A  man  of  middle  years,  athletic  build,  blond  hair,  blue  eyes,  and  a  smile  met  me.  Pete  was  on  his 
was  across  the  hall  to  talk  to  one  of  four  secretaries  who  work  at  IMS.  "Go  on  in  and  have  a  seat,  I'll  be 
with  you  in  a  sec"  and  Pete  was  off  leaving  me  to  ponder  what  I  was  volunteering  for. 

Pete's  leadership  style  is  participatory.   Sharon,  a  media  specialist,  summed  it  up  this  way  when 
we  were  setting  up  a  new  media  center,  "People  respect  Pete  because  there  isn't  anything  he  won't  do. 
Today  for  example  he's  helping  put  up  shelves  and  shelving  books.  You  don't  find  many  administrators 
willing  to  do  that."  When  ask  about  what  he  expects  from  employees,  Pete  responded,  "I  try  to  be  a  model 
in  my  work.  I  model  what  I  want  them  to  be  doing  for  me  [library  and  technology  related]  ...  I  don't 
expect  anything  more  from  them  than  I  do  from  myself." 

Pete  describes  his  position  in  the  following  way,  "I'm  a  facilitator.  I  get  people  together  in  order 
to  make,  things  happen.  I  help  solve  problems.  I  look  ahead  to  our  needs."  He  is  described  by  most,  all  of 
the  elementary  media  specialists  as  a  "good  leader"  (some  say  boss)  and  those  answering  always  added  "I 
like  Pete."    What  is  a  good  leader  /  boss?  To  the  media  specialists  I  spoke  with  it  means  someone  they 
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can  count  on  when  they  are  in  a  pinch,  someone  to  help  them  keep  up  in  their  profession,  someone  to 
brainstorm  with,  someone  to  help  provide  direction,  and  someone  to  talk  things  over  with. 

The  media  specialists  like  Pete  as  a  person,  but  as  Heller,  Van  Til,  &  Zurcher  ( 1986)  points  out  a 
leader's  "authority  must  be  defined  in  terms  of  the  follower's  acceptance  of  the  legitimacy  of  the  influence 
attempt"  (p.  6).   From  conversations  with  the  media  specialists,  the  following  concerns  were  voiced.  "He's 
always  so  busy.  I  don't  call  him  anymore  unless  it's  a  real  problem."  "Pete  has  been  an  administrator  for 
so  long  he  doesn't  really  know  what's  happening  in  the  schools,  or  what  we  really  do."  "I  think  Pete  has 
gotten  his  head  bumped  once  too  often  and  has  pulled  back  we  don't  try  enough  different  ways  or  things 
anymore."  Nested  within  these  comments  are  their  concerns  regarding  Pete's  leadership. 

"Leader  and  follower  must  be  united  by  common  goals  and  aspirations  and  by  a  will  to  lead,  on 
one  side,  and  a  will  to  follow  on  the  other.  The  leader  must  be  a  member  of  the  group,  and  must  share  its 
norms,  its  objectives,  and  its  aspirations"  (Heller,  Van  Til,  &  Zurcher,  1986,  p.6).  He  is  described  by  most 
all  of  the  elementary  media  specialists  as  a  "good  leader."  However,  Pete's  relationship  with  the  secondary 
media  specialists  is  very  different. 

The  secondary  media  specialists  are  not  willing  followers  of  Pete.  For  example,  Laura  did  not 
view  Pete  as  her  supervisor,  "He  is  a  coordinator  and  a  facilitator  for  the  secondary  people.  My  supervisor 
and  boss  is  my  principal."  Pete  believes  he  is  more  than  what  she  described  but  remains  optimistic, 
"Remember,  they  didn't  have  anyone  in  my  position  before  I  came  on  board  [as  director].  So  for  some  of 
those  people  it  has  been  a  real  adjustment  [to  view  him  as  a  leader  and  not  a  member]. .  . .  That  will 
change.  You  evolve.  Some  of  those  people  will  retire  and  you  hire  others  to  fit  more  in  the  niche." 
Pete's  philosophy  coincides  with  Heller,  Van  Til,  &  Zurcher's  postulate  (1986,  p.  254),  "Good  leadership 
enhances  followers,  just  as  good  followership  enhances  leaders."  By  attempting  to  hire  personnel  in  line 
with  his  values  and  beliefs,  Pete  is  strengthening  his  position  and  programs  within  the  district. 

Most  days  meetings  take  up  70  percent  of  Pete's  day.  He  meets  with  "his  boss  and  his  boss' 
boss,"  with  principals,  with  vendors,  with  media  specialists,  with  coordinators,  with  teachers,  with 
University  representatives,  with  partners  in  education  representatives,  with  parents,  with  students,  and  with 
a  variety  of  "outsiders"  who  are  interested  in  BPS'  technology.  "My  job  is  meetings!"  Pete  proclaimed 
one  day  after  an  unusually  long  meeting. 

Meetings  are  bureaucratic  by  nature,  but  there  is  no  way  around  them  in  any  social  structure. 
However,  Pete  attends  meetings  with  the  common  good  of  the  district  at  the  top  of  his  agenda.  He  goes 
into  the  meetings  with  the  plan  to  do  what  is  best  for  the  district  or  department  based  on  what  the  students 
of  the  district  need  or  deserve.  Pete  feels  he  has  a  stake  in  the  educational  process  as  he  has  two  daughters 
in  the  district.  Before  he  makes  any  decision  or  votes  on  any  action,  Pete  stated  he  asks  himself  "How 
will  this  help  the  students,  the  district? 

Pete's  technology  leadership  is  on  cue  with  the  literature  regarding  his  creativity,  his  technical 
know  how,  and  his  visionary  focus  for  the  district  (Kearsley  &  Lynch,  1992).  A  result  of  Pete's  visionary 
skills  is  now  tangible  in  all  buildings.  He  believed  card  catalogs  were  obsolete  and  should  be  replaced  with 
automation.  Every  media  center  in  the  district  has  an  automated  card  catalog  for  students  and  faculty.  The 
students  and  faculty  I  observed  were  very  comfortable  with  the  technology  and  the  specialists  say  the 
students  use  it  more  than  they  ever  did  the  card  catalogs.  In  addition,  the  specialist  state  students  are  finding 
and  using  more  resources  for  their  classroom  reports.  Another  visionary  resource  that  is  visible  in  all  of 
the  schools  is  CD-ROM  workstations.  Pete  believes  students  will  be  required  to  remember  less  facts  but  to 
know  were  to  find  the  key-information  they  need  just  as  adults  are  now  doing  in  the  workplace.  These  are 
not  truly  revolutionary  ideas  because  these  resources  have  been  available  to  the  adult  population  for  some 
time,  but  are  an  "evolution"  in  order  to  keep  up  with  the  outside  world.  BPS  is  one  of  the  few  school 
districts  that  has  put  these  resources  into  all  buildings,  not  just  magnet  schools. 

Literature  Review:  Communication 

Each  person's  life  is  lived  as  a  series  of  conversations.  Tannen  (1990)  states  men's 
conversational  style  reflects  their  attitudes  regarding  life;  it  is  a  contest,  a  struggle  to  preserve  independence 
and  avoid  failure.  Conversations  are  negotiations  in  which  people  try  to  achieve  and  maintain  the  upper 
hand  if  they  can.  Men's  conversations  are  efforts  to  protect  themselves  from  others'  attempt  to  put  them 
down  and  push  them  around. 

Like  men,  women's  conversational  style  reflect  their  own  attitude  towards  life;  it  is  a  struggle  to 
preserve  intimacy  and  avoid  isolation.  Conversations  are  negotiations  for  closeness  in  which  people  try  to 
seek  and  give  confirmation  and  support,  and  to  reach  consensus  (Tannen,  1990).  Women  attempt  to  protect 
themselves  from  others'  attempts  to  push  them  away. 
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Each  style  is  valid  on  its  own  terms,  misunderstandings  arise  because  the  styles  are  different.  We 
all  want,  above  all,  to  be  heard,  but  not  merely  to  be  heard.  We  want  to  be  understood,  heard  for  what  v/e 
think  we  arc  saying  and  for  what  we  know  we  meant.  Women  and  men  are  both  often  frustrated  by  other's 
way  of  responding  to  their  expression  of  troubles. 

Many  men  see  themselves  as  problem  solvers  and  a  complaint  or  trouble  is  a  challenge  to  their 
ablity  to  find  a  solution.   If  women  resent  men's  tendency  to  offer  solutions  to  problems,  men  complain 
about  women's  refusal  to  take  action  to  solve  problems  they  complain  about.  Many  women  who 
habitually  report  problems  at  work  or  in  friendships,  the  message  is  not  one  of  complaint  but  is  a  bid  for  an 
expression  of  understanding  -  wc  are  the  same,  you  are  not  alone  (Tannen,  1990). 

The  Way  I  Hear  It 

Deborah  Tannen  (1990)  states  women  and  men  have  characteristically  different  conversational 
styles  and  feels  thai  gender  differences  account  for  these  contrasting  styles.  At  all  ages,  females  are  more, 
likely  to  phrase  their  preferences  as  suggestions,  appearing  to  give  others  options  in  deciding  what  to  do. 
In  addition,  women  try  to  influence  others  by  making  polite  suggestions.  Men  use  more  direct  commands 
in  most  all  situations  (Tannen,  1990).  It  is  these  differences  in  styles  that  give  rise  to  many  female/male 
conflicts.  People  with  direct  styles  of  conversation,  typically  men,  perceive  indirect  requests,  typically 
women,  as  manipulative  or  whiny. 

The  direct  conversation  style  is  very  evident  with  Pete  in  all  methods  of  communication.  Pete  is 
brief  and  direct  with  his  conversations,  having  few  non-business  conversations.  For  meetings  he  is 
responsible  for  running,  Pete  always  has  a  typed  outline  to  hand  out  to  the  members  and  he  follows  it  very 
closely.  Pete  states  he  has  to  follow  agendas  in  order  to  keep  meetings  on  track  and  on  time.  Many  times 
Pete  has  meetings  back  to  back  and  can  not  afford  time  over  runs. 

In  professional  phone  conversations  Pete  is  very  consistent  in  his  delivery,  he  introduces  himself 
(Hello,  this  is  Pete  Young-he  will  add  the  Dr.  title  when  dealing  with  people  outside  of  the  district)  and 
follows  with  "I'm  calling  about .  . .  ."  He  will,  after  the  business  has  been  conducted  and  depending  on  his 
schedule,  make  small  talk  for  a  few  minutes  with  the  person  on  the  other  end  of  the  line.  He  concludes  his 
conversations  with  "Well-thanks  a  lot.  Bye." 

In  conversations  with  media  specialists  regarding  the  quality  of  their  communication  with  Pete  I 
was  told  he  was  a  "very  good  listener,  most  of  the  time."  However,  one  specialist  added  the  following:  "He 
is  a  male  boss  and  he  doesn't  understand  where  we're  coming  from  (female  perspective).  He  doesn't  deal 
well  with  the  whincrs." 

Pete's  main  complaint  regarding  other  district  personnel,  including  several  of  the  media  specialists 
(all  media  specialists  were  female),  was  their  socializing,  "They're  always  entertaining  someone."  Tannen 
(1990)  explains  women  mix  business  and  personal  talk  to  establish  a  comfortable  working  relationship  and 
are  thus  viewed  as  socializing  by  those  with  the  direct  approach.  Women  believe  the  personal  talk  makes  it 
possible  for  them  to  conduct  their  business  successfully  and  efficiently  (Tannen,  1 990). 

Indirect  methods  of  communication  that  act  to  nurture  relationships  are  often  viewed  as  socializing 
and  therefore  those  individuals  are  perceived  as  not  doing  their  job  or  as  not  taking  their  job  seriously  by 
people  who  use  the  direct  method.  An  example  of  communication  methods  creating  incorrect  assumptions 
about  another  individual  involves  Laura,  a  secondary  media  specialist  and  Pete.  I  observed  Laura  giving  a 
tour  to  visitors  from  another  school  district.  She  showed  them  around  the  media  center  at  the  school  she 
was  employed  at,  pointed  out  the  various  technologies  available  and  the  reasons  the  district  supported  those 
technologies  in  the  schools.  Laura  addressed  their  questions  regarding  all  aspects  of  technology  used  and 
those  she  could  not  answer  encouraged  them  to  call  and  talk  with  Pete.  She  was  an  excellent  ambassador 
for  the  district  and  for  the  use  of  technology  in  school  media  centers.  However,  Pete  perceives  Laura  as  a 
socialite  and  as  someone  who  "just  sits  in  her  office  waiting  to  retire." 

Observations 

A  media  director  should  be  an  administrative  position  who  orchestrates  the  use  of  technology 
within  the  school  district.  The  duties  and  responsibilities  of  the  media  director  evolve  as  the  district's 
climate  and  resources  do.  Regarding  his  own  definition  of  his  position,  Pete  fluctuates.  For  example, 
when  asked  what  it  is  he  does,  the  first  descriptor  Pete  uses  is  facilitator,  yet,  he  is  frustrated  by  the 
secondary  media  specialists  for  viewing  him  in  this  light. 

Pete  is  a  leader,  promoted  from  the  ranks  of  media  specialists.  However,  his  administrative  duties 
have  caused  him  some  "followship"  difficulties  and  he  has  opted  to  "wait  them  [media  specialists]  out."  A 
leader's  legitimacy  is  defined  by  the  followers'  acceptance  of  him  as  a  leader  and  of  him  as  a  part  of  the 
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group.  As  Pete  waits  for  his  position  to  become  legitimate,  he  becomes  less  a  member  of  the  group  he 
leads. 

As  director,  Pete  must  have  effective  communication  skills.  However  without  conscientiously 
realizing  it,  gender  has  played  a  part  in  his  attitude  and  dealings  with  several  of  the  media  specialists.  Pete 
uses  the  direct  method  of  communication,  which  is  typical  of  men.  Most  of  the  medial  specialists  used  the 
indirect  approach,  which  is  typical  of  women.  Pete's  perceptions  of  many  of  the  media  specialists  as 
"socialites"  and  "time  wasters"  are  a  direct  result  of  the  two  modes  of  communications. 
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Ethnography 

An  ethnography  can  be  defined  as  an  in-depth  analytical  description  of  an  intact  cultural  scene. 
This  research  observes  human  behavior  in  its  natural  setting  (holistic  inquiry  carried  out  in  a  natural  setting) 
over  a  period  of  time.  Naturalistic  perspective  emphasize  humans  as  the  primary  data  gathering  instrument 
and  the  use  of  triangulated  data-gathering  procedures  (Goetz  &  LeCompte,  1984).  Human  observation  is 
flexible  and  can  adapt  to  a  complex  situation  as  it  evolves.  In  addition,  human  observation  allows  for  the 
identification  of  biases  that  result  from  interactions  and  value  differences  between  the  "instrument"  and  the 
subject  (Borg,  1989).  To  help  reduce  biases,  researchers 

should  use  triangulation  of  data.  Triangulation  is  the  process  of  collecting  and  examining  related  documents 
to  serve  as  crossreferences  to  observed  data.  Researchers  using  naturalistic  inquiry  purposely  select  their 
subject/group  or  setting.  By  purposeful  selection,  a  qualitative  researcher  will  be  more  likely  to  uncover 
the  full  array  of  "multiple  realities"  relevant  to  an  inquiry.  The  focus  of  the  ethnographer's  inquiry  is  on  the 
mundane,  everyday  practices  of  people.  The  paramount  objective  of  qualitative  research  is  understanding 
rather  than  generalizing  or  identifying  cause  and  effect  (Whitt,  1991). 

Pilot  Study 
Research  Design 

This  study  was  undertaken  to  explore  the  leadership  qualities  of  a  district  media  director.  The  school 
district,  Byrdsville  Public  School  District  or  BPS  as  school  personnel  referred  to  it  (This  is  a  pseudonym 
for  the  actual  school  district  for  confidentiality.),  is  a  mid-sized  school  district  with  26  school  buildings  and 
approximately  13,000  students.  Using  ethnographic  research  methods,  the  following  questions  were 
investigated: 

What  are  the  beliefs  and  values  which  underlies  this  media  director's  decisions? 

What  are  the  recurring  events  and  practices  of  the  media  director? 

What  are  the  relationships  that  media  director  must  cultivate? 

What  are  the  day-to-day  encounters  of  a  media  director? 

What  are  the  leadership  qualities  of  this  media  director? 
The  significance  of  this  study  revolves  around  the  lack  of  any  research  on  media  directors.  Research  has 
been  typically  focused  on  either  administrators  (superintendents  or  principals)  or  teachers. 

Data  for  this  study  were  obtained  through  observation,  interviews,  and  artifact  analysis. 
Respondents  were  guaranteed  confidentiality  and  all  data  presented  in  this  study  are  designed  to  protect  their 
anonymity. 

Roles  and  Experiences  of  Investigator 

During  this  study,  I  became  a  participant  observer,  an  intern  to  the  media  director.  My  internship 
was  contracted  as  one  day  a  week,  Tuesday,  for  two  semesters  so  I  could  observe  a  full  school  year  cycle.  In 
addition,  I  attended  special  events  I  was  invited  to.  By  being  actively  involved  in  the  situation  being 
observed,  insights  and  interpersonal  relationships  developed  which  would  have  been  virtually  impossible  to 
achieve  through  any  other  method.  This  role,  participant  observer,  requires  the  researcher  to  come  to  the 
situation  as  a  learner  and  to  try,  at  least  partially,  to  be  socialized  into  the  group.  The  socialization  process 
allows  the  researcher  to  experience  the  physical,  relational,  and  emotional  rea.itics  of  the  setting  as  it  exists 
for  the  members  (Dobbert,  1984). 

In  the  beginning  as  a  participant  observer,  Dobbert  (1984)  recommends  the  researcher  keep  in  the 
background,  keep  quiet,  watch,  and  listen.  In  addition,  the  researcher  should  be  maintaining  a  friendly  and 
willing  stance,  offering  to  assist  with  single  or  onerous  tasks,  accepting  invitations  and  assistance 
gratefully,  asking  questions  when  you  don't  think  it  would  be  bothersome,  and  trying  to  learn  the  language 
of  the  group  (Dobbert,  1984). 

Wolcott  (1988)  warns  that  taking  a  more  active  role  than  the  "observer"  has  costs  attached; 
research  efforts  are  sometimes  secondary  to  the  participation.  However,  the  benefits  of  being  a  participant- 
observer  includes:  1)  quality  of  direct  on-site  observation;  with  the  participant,  long  term  observation  and 
group  acceptance  is  more  likely  to  occur;  2)  freedom  of  access;  more  documents  and  subjects  are  available  to 
the  researcher  when  she  or  he  is  a  "part"  of  the  group;  3)  intensity  of  observation;  by  being  in  the  "loop" 
there  are  typically  more  hours  of  observation,  increasing  the  chances  of  observing  the  common  and  the 
exceptional;  and  4)  data  sampling;  by  being  a  participant,  the  researcher  may  be  included  in  meetings 
(formal  and  informal)  and  be  included  in  mailings  that  she  or  he  would  not  have  known  about  otherwise 
(Borg,  1989). 
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Data  Collection  Methods 

"Ethnographers  consider  data  to  be  potentially  verifiable  information  obtained  from  the 
environment.  The  problem  of  data  collection,  tk;n,  consists  of  defining  appropriate  information  and 
developing  strategies  for  obtaining  it"  (Goetz  &  LeCompte,  1984,  p.  107).  The  most  common  data 
collection  includes  observation,  interviewing,  researcher-designed  instruments,  and  content  analysis  of 
human  artifacts  (Goetz  &  LeCompte,  1984). 

Interactions  and  observations  were  the  primary  sources  for  the  field  notes  in  this  study.  Within 
these  activities  are  discussions  relative  to  the  media  director's  position.  With  each  observation  I  became 
more  at  ease  discriminating  between  what  should  and  should  not  be  recorded,  knowing  what  I  left  out  shaped 
my  data  as  certainly  as  that  which  I  included.  During  observations  I  would  jot  down  notes  and  quotes. 
Later,  these  would  be  rewritten,  expanding  my  description  of  the  experience. 

In  addition  to  the  field  notes,  informal  interviews  were  conducted  with  the  media  director, 
secretaries,  media  specialists,  and  principals.  I  used  interviewing  methods  to  "get  at"  the  reasons  for  certain 
actions  I  had  observed  or  "get  to"  the  individual's  opinion,  background,  and  experiences.  These  interviews 
ensured  a  variety  of  perspectives  and  consistency  of  data. 

Correspondence  or  documents  relating  to  the  media  director  were  collected  in  efforts  to  cross- 
reference  to  the  observed  data.  These  artifacts  were  typical  correspondence  of  the  media  director,  letters  and 
memos  that  were  sent  within  and  outside  of  the  district.  In  addition,  I  collected  original  material  Pete 
created  for  specific  groups  (media  specialists,  secretaries,  district  coordinators,  district  technology  group). 
This  process  allowed  for  triangulation  and  a  "trustworthiness"  of  the  data  (Lincoln  &  Guba,  1985). 

Respondents 

Following  Lincoln  &  Guba's  (1985)  suggestion,  purposeful  sampling  was  employed  in  order  to 
obtain  as  much  information  as  possible  and  to  get  the  maximum  variation  in  sampling.  Further,  the 
intended  purpose  of  data  collection  was  not  to  generate  enough  data  to  be  able  to  draw  conclusions  or 
generalizations  but  to  be  able  to  provide  results  with  the  uniqueness  of  the  subject  or  phenomenon  (Lincoln 
&  Guba,  1985). 

All  respondents  in  this  pilot  study  were  employed  by  Byrdsville  Public  School  District.  Every 
respondent  was  told  they  would  have  anonymity  and  all  information  would  be  confidential.  Furthermore, 
each  respondent  was  given  a  pseudonym  either  by  choosing  it  herself/himself  or  one  was  provided  by  the 
researcher  in  order  to  maintain  confidentiality.  Respondents  who  volunteered  to  be  observed  and  to 
interviews  included: 

Media  director:  Dr.  Pete  Young 

Microcomputer  coordinator:  Bob 

Media  specialists:  Sharon,  Laura,  Brenda 

Principal:  Paul 

Secretaries:  Nancy,  Elizabeth,  Frances,  Susan 
In  addition,  there  were  many  individuals  who  were  a  part  of  the  project  because  of  their  interactions  with 
Pete.  I  was  introduced  as  "his  intern  who  is  learning  how  a  media  program  works." 

Analysis 

Data  does  not  answer  research  questions;  it  must  be  analyzed  before  a  scientific  perspective  can  be 
offered.  Lincoln  &  Guba  (1985)  promote  the  induction  and  constant  comparison  method  of  data  analysis. 
Dobbert's  (1984)  description  of  Darwinian  natural  history  method  follows  this  same  research  strategy.  The 
method  includes  five  steps  and  is  a  spiral  process.  The  five  steps  are:  1)  careful  observation,  2)  have  a 
conceptual  scheme  into  which  to  try  and  fit  the  data  (this  occurs  with  step  1),  3)  change  the  concepts, 
assumptions,  or  hypotheses  to  accommodate  the  facts  (this  occurs  after  step  1),  4)  group  facts  by  categories 
and  arrive  at  conclusions  by  explaining  the  relationships  or  processes  that  create  the  differences  and 
similarities  expressed  in  the  categories  (this  occurs  with  step  3),  and  5)  turn  to  a  new  observation  directed 
by  the  discoveries  stemming  from  the  analytic  process  as  carried  out  through  the  previous  four  steps 
(Dobbert,  1984). 

Data  were  analyzed  and  categorized  into  issues  which  describe  the  inter-relatedness  of  theory  and  the 
decision  making  /  actions  of  the  media  director.  The  themes  of  the  study  are  presented  in  the  following  two 
sections:  1)  Leadership:  Lead,  Follow,  or  Please  Retire!,  2)  Communication:  The  Way  I  Hear  It. 

Literature  Review:  Leadership 

Leadership  is  a  process  whereby  one  person  exerts  social  influence  over  the  members  of  the  group. 
A  leader  is  a  person  with  power  over  others  who  exercises  this  power  for  the  purpose  of  influencing  their 
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behavior  (Heller,  Van  Till  &  Zurcher,  1986).  Today,  leaders  are  challenged  as  they  find  themselves  working 
with  more  individuals  who  are  different  from  themselves  and  they  must  bridge  the  empathy  gap.  A  leader's 
creativity  must  be  used  when  working  with  the  resources  of  their  diverse  team  members  and  avoid 
neglecting  the  team  members'  important  skills  and  competencies. 

Educational  technology  leadership  shares  much  with  the  research  on  leadership.  However,  the 
specific  knowledge  and  abilities  connected  with  technology  provides  for  unique  considerations.  An  analysis 
of  the  effectiveness  of  technology  in  education  suggests  that  the  manner  in  which  technology  is 
implemented  is  more  important  than  any  intrinsic  characteristics  of  the  technology  (Kearsley  &  Lynch, 
1992).  Leadership  plays  a  very  critical  role  in  the  process  of  the  adoption  of  technology. 

Technology  leaders  today  are  expected  to  possess  strong  visionary  skills,  have  excellent 
interpersonal  and  communication  skills,  as  well  as  a  measure  of  technical  competence  in  addition  to  the 
"basic  leadership"  capabilities  (Ray,  1992).  To  be  successful,  these  leaders  must  be  successful  in  their 
abilities  to  articulate  and  influence  cultural  norms  and  values;  they  are  expected  to  shape  the  culture  of  the 
school  by  creating  a  new  vision  that  members  can  believe  in  and  act  upon  (Kearsley  &  Lynch,  1992). 

Electronic  technology  has  been  viewed  as  a  radical  departure  from  the  traditional  educational 
technologies  (books,  chalkboards,  duplicating  machines,  teaching  systems,  ect.).  The  new  technology  has 
only  been  widely  available  or  affordable  since  the  mid  1980's.  Therefore,  the  majority  of  teachers  did  not 
have  the  opportunity  to  practice  with  electronic  technology  during  their  preprofessional  training.  Teachers 
are  typically  insecure  and  unskilled  in  regards  to  using  technology  in  their  classroom  and  mandates  to  use 
technology  by  administrators  only  leave  them  frustrated  and  floundering 

The  failure  of  any  technology  can  result  for  a  variety  of  reasons.  Some  of  these  reasons  can 
include  lack  of  teacher  training,  lack  of  appropriate  materials/equipment,  poorly  conceived  implementation 
plans,  unrealistic  goals/expectations,  or  insufficient  funding.  Most  of  these  pitfalls  can  be  corrected  with 
good  technological  leadership.  Technology  leaders  must  identify  educational  problems  which  technology 
can  solve  and  build  theoretical,  political,  and  financial  support  structures  to  ensure  success  (Kearsley  & 
Lynch,  1992). 

Lead,  Follow,  or  Please  Retire 

The  first  time  I  was  in  Pete's  office,  was  to  discuss  my  internship  with  him.  Pete's  office  is 
located  in  an  old  brick  school  building  called  Smith  by  the  district.  Smith  is  located  in  the  center  of  the 
city  and  is  surrounded  by  federal  assistance  housing  projects.  The  east  side  of  Smith  is  its  architectural 
front  and  has  a  large  green  yard  between  it  and  a  major  street  The  drive  entrance  is  located  behind  the 
building  with  a  small  paved  parking  lot  at  the  south  end  of  the  building  and  a  six  foot  fence  on  the 
perimeter  of  the  property.  The  doors  coming  into  the  building  from  the  parking  lot  are  the  doors  used 
primarily  by  the  Instructional  Materials  Services  (IMS)  department.  The  other  half  of  the  building  is 
occupied  by  the  alternative  high  school.  A  high  school  for  students  who  could  or  would  not  "fit"  into  the 
district's  traditional  high  schools. 

Upon  entering  from  the  parking  lot  there  is  a  hundred  yard  walk  down  a  dimly  light  hallway  filled 
with  furniture  and/or  equipment,  depending  of  the  current  activities  of  IMS,  to  Pete's  office.  His  office 
reflects  its  age  by  the  height  of  the  ceiling,  the  light  fixtures,  exposed  plumbing,  and  dilapidated  walls.  The 
rectangular  office  is  painted  yellow.  Yellow  is  considered  a  positive,  warm  color;  it  is  also  the  color 
disliked  by  the  most  people  (Raines,  1989).  With  his  desk  facing  the  entrance  of  the  office,  there  are  four, 
six  foot  high  windows  behind  him  that  create  a  halo  effect  with  the  afternoon  sunshine.  A  round  table  and 
four  chairs  occupy  the  front  of  the  office. 

A  man  of  middle  years,  athletic  build,  blond  hair,  blue  eyes,  and  a  smile  met  me.  Pete  was  on  his 
was  across  the  hall  to  talk  to  one  of  four  secretaries  who  work  at  IMS.  "Go  on  in  and  have  a  seat,  I'll  be 
with  you  in  a  sec"  and  Pete  was  off  leaving  me  to  ponder  what  I  was  volunteering  for. 

Pete's  leadership  style  is  participatory.  Sharon,  a  media  specialist,  summed  it  up  this  v/ay  when 
we  were  setting  up  a  new  media  center,  "People  respect  Pete  because  there  isn't  anything  he  won't  do. 
Today  for  example  he's  helping  put  up  shelves  and  shelving  books.  You  don't  find  many  administrators 
willing  to  do  that."  When  ask  about  what  he  expects  from  employees,  Pete  responded,  "I  try  to  be  a  model 
in  my  work.  I  model  what  I  want  them  to  be  doing  for  me  [library  and  technology  related]  ...  I  don't 
expect  anything  more  from  them  than  I  do  from  myself." 

Pete  describes  his  position  in  the  following  way,  "I'm  a  facilitator.  I  get  people  together  in  order 
to  make  things  happen.  I  help  solve  problems.  I  look  ahead  to  our  needs."  He  is  described  by  most  all  of 
the  elementary  media  specialists  as  a  "good  leader"  (some  say  boss)  and  those  answering  always  added  "I 
like  Pete."    What  is  a  good  leader  /  boss?  To  the  media  specialists  I  spoke  with  it  means  someone  they 
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can  count  on  when  they  are  in  a  pinch,  someone  to  help  them  keep  up  in  their  profession,  someone  to 
brainstorm  with,  someone  to  help  provide  direction,  and  someone  to  talk  things  over  with. 

The  media  specialists  like  Pete  as  a  person,  but  as  Heller,  Van  Til,  &  Zurcher  (1986)  points  out  a 
leader's  "authority  must  be  defined  in  terms  of  the  follower's  acceptance  of  the  legitimacy  of  the  influence 
attempt"  (p.  6).  From  conversations  with  the  media  specialists,  the  following  concerns  were  voiced.  "He's 
always  so  busy.  I  don't  call  him  anymore  unless  it's  a  real  problem."  "Pete  has  been  an  administrator  for 
so  long  he  doesn't  really  know  what's  happening  in  the  schools,  or  what  we  really  do."  "I  think  Pete  has 
gotten  his  head  bumped  once  too  often  and  has  pulled  back  we  don't  try  enough  different  ways  or  things 
anymore."  Nested  within  these  comments  are  their  concerns  regarding  Pete's  leadership. 

"Leader  and  follower  must  be  united  by  common  goals  and  aspirations  and  by  a  will  to  lead,  on 
one  side,  and  a  will  to  follow  on  the  other.  The  leader  must  be  a  member  of  the  group,  and  must  share  its 
norms,  its  objectives,  and  its  aspirations"  (Heller,  Van  Til,  &  Zurcher,  1986,  p.6).  He  is  described  by  most 
all  of  the  elementary  media  specialists  as  a  "good  leader."  However,  Pete's  relationship  with  the  secondary 
media  specialists  is  very  different. 

The  secondary  media  specialists  are  not  willing  followers  of  Pete.  For  example,  Laura  did  not 
view  Pete  as  her  supervisor,  "He  is  a  coordinator  and  a  facilitator  for  the  secondary  people.  My  supervisor 
and  boss  is  my  principal."  Pete  believes  he  is  more  than  what  she  described  but  remains  optimistic, 
"Remember,  they  didn't  have  anyone  in  my  position  before  I  came  on  board  [as  director].  So  for  some  of 
those  people  it  has  been  a  real  adjustment  [to  view  him  as  a  leader  and  not  a  member].  . . .  That  win 
change.  You  evolve.  Some  of  those  people  will  retire  and  you  hire  others  to  fit  more  in  the  niche." 
Pete's  philosophy  coincides  with  Heller,  Van  Til,  &  Zurcher's  postulate  (1986,  p.  254),  "Good  leadership 
enhances  followers,  just  as  good  followership  enhances  leaders."  By  attempting  to  hire  personnel  in  line 
with  his  values  and  beliefs,  Pete  is  strengthening  his  position  and  programs  within  the  district. 

Most  days  meetings  take  up  70  percent  of  Pete's  day.  He  meets  with  "his  boss  and  his  boss' 
boss,"  with  principals,  with  vendors,  with  media  specialists,  with  coordinators,  with  teachers,  with 
University  representatives,  with  partners  in  education  representatives,  with  parents,  with  students,  and  with 
a  variety  of  "outsiders"  who  are  interested  in  BPS' technology.  "My  job  is  meetings!"  Pete  proclaimed 
one  day  after  an  unusually  long  meeting. 

Meetings  are  bureaucratic  by  nature,  but  there  is  no  way  around  them  in  any  social  structure. 
However,  Pete  attends  meetings  with  the  common  good  of  the  district  at  the  top  of  his  agenda.  He  goes 
into  the  meetings  with  the  plan  to  do  what  is  best  for  the  district  or  department  based  on  what  the  students 
of  the  district  need  or  deserve.  Pete  feels  he  has  a  stake  in  the  educational  process  as  he  has  two  daughters 
in  the  district.  Before  he  makes  any  decision  or  votes  on  any  action,  Pete  stated  he  asks  himself  "How 
will  this  help  the  students,  the  district? 

Pete's  technology  leadership  is  on  cue  with  the  literature  regarding  his  creativity,  his  technical 
know  how,  and  his  visionary  focus  for  the  district  (Kearsley  &  Lynch,  1992).  A  result  of  Pete's  visionary 
skills  is  now  tangible  in  all  buildings.  He  believed  card  catalogs  were  obsolete  and  should  be  replaced  with 
automation.  Every  media  center  in  the  district  has  an  automated  card  catalog  for  students  and  faculty.  The 
students  and  faculty  I  observed  were  very  comfortable  with  the  technology  and  the  specialists  say  the 
students  use  it  more  than  they  ever  did  the  card  catalogs.  In  addition,  the  specialist  state  students  are  finding 
and  using  more  resources  for  their  classroom  reports.  Another  visionary  resource  that  is  visible  in  all  of 
the  schools  is  CD-ROM  workstations.  Pete  believes  students  will  be  required  to  remember  less  facts  but  to 
know  were  to  find  the  key-information  they  need  just  as  adults  are  now  doing  in  the  workplace.  These  are 
not  truly  revolutionary  ideas  because  these  resources  have  been  available  to  the  adult  population  for  some 
time,  but  are  an  "evolution"  in  order  to  keep  up  with  the  outside  world.  BPS  is  one  of  the  few  school 
districts  that  has  put  these  resources  into  all  buildings,  not  just  magnet  schools. 

Literature  Review:  Communication 

Each  person's  life  is  lived  as  a  series  of  conversations.  Tannen  (1990)  states  men's 
conversational  style  reflects  their  attitudes  regarding  life;  it  is  a  contest,  a  struggle  to  preserve  independence 
and  avoid  failure.  Conversations  are  negotiations  in  which  people  try  to  achieve  and  maintain  the  upper 
hand  if  they  can.  Men's  conversations  are  efforts  to  protect  themselves  from  others'  attempt  to  put  them 
down  and  push  them  around. 

Like  men,  women's  conversational  style  reflect  their  own  attitude  towards  life;  it  is  a  struggle  to 
preserve  intimacy  and  avoid  isolation.  Conversations  are  negotiations  for  closeness  in  which  people  try  to 
seek  and  give  confirmation  and  support,  and  to  reach  consensus  (Tannen,  1990).  Women  attempt  to  protect 
themselves  from  others'  attempts  to  push  them  away. 
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Each  style  is  valid  on  its  own  terms,  misunderstandings  arise  because  the  styles  are  different.  We 
all  want,  above  all,  to  be  heard,  but  not  merely  to  be  heard.  We  want  to  be  understood,  heard  for  what  we 
think  we  are  saying  and  for  what  we  know  we  meant.  Women  and  men  are  both  often  frustrated  by  other's 
way  of  responding  to  their  expression  of  troubles. 

Many  men  see  themselves  as  problem  solvers  and  a  complaint  or  trouble  is  a  challenge  to  their 
ablity  to  find  a  solution.  If  women  resent  men's  tendency  to  offer  solutions  to  problems,  men  complain 
about  women's  refusal  to  take  action  to  solve  problems  they  complain  about.  Many  women  who 
habitually  report  problems  at  work  or  in  friendships,  the  message  is  not  one  of  complaint  but  is  a  bid  for  an 
expression  of  understanding  -  we  are  the  same,  you  are  not  alone  (Tannen,  1990). 

The  Way  I  Hear  It 

Deborah  Tannen  (1990)  states  women  and  men  have  characteristically  different  conversational 
styles  and  feels  that  gender  differences  account  for  these  contrasting  styles.  At  all  ages,  females  are  more 
likely  to  phrase  their  preferences  as  suggestions,  appearing  to  give  others  options  in  deciding  what  to  do. 
In  addition,  women  try  to  influence  others  by  making  polite  suggestions.  Men  use  more  direct  commands 
in  most  all  situations  (Tannen,  1990).  It  is  these  differences  in  styles  that  give  rise  to  many  female/male 
conflicts.  People  with  direct  styles  of  conversation,  typically  men,  perceive  indirect  requests,  typically 
women,  as  manipulative  or  whiny. 

The  direct  conversation  style  is  very  evident  with  Pete  in  all  methods  of  communication.  Pete  is 
brief  and  direct  with  his  conversations,  having  few  non-business  conversations.  For  meetings  he  is 
responsible  for  running,  Pete  always  has  a  typed  outline  to  hand  out  to  the  members  and  he  follows  it  very 
closely.  Pete  states  he  has  to  follow  agendas  in  order  to  keep  meetings  on  track  and  on  time.  Many  times 
Pete  has  meetings  back  to  back  and  can  not  afford  time  over  runs. 

In  professional  phone  conversations  Pete  is  very  consistent  in  his  delivery,  he  introduces  himself 
(Hello,  this  is  Pete  Young-he  will  add  the  Dr.  title  when  dealing  with  people  outside  of  the  district)  and 
follows  with  Tm  calling  about . . . ."  He  will,  after  the  business  has  been  conducted  and  depending  on  his 
schedule,  make  small  talk  for  a  few  minutes  with  the  person  on  the  other  end  of  the  line.  He  concludes  his 
conversations  with  "Well-thanks  a  lot.  Bye." 

In  conversations  with  media  specialists  regarding  the  quality  of  their  communication  with  Pete  I 
was  told  he  was  a  "very  good  listener,  most  of  the  time."  However,  one  specialist  added  the  following:  "He 
is  a  male  boss  and  he  doesn't  understand  where  we're  coming  from  (female  perspective).  He  doesn't  deal 
well  with  the  whiners." 

Pete's  main  complaint  regarding  other  district  personnel,  including  several  of  the  media  specialists 
(all  media  specialists  were  female),  was  their  socializing,  "They're  always  entertaining  someone."  Tannen 
(1990)  explains  women  mix  business  and  personal  talk  to  establish  a  comfortable  working  relationship  and 
are  thus  viewed  as  socializing  by  those  with  the  direct  approach.  Women  believe  the  personal  talk  makes  it 
possible  for  them  to  conduct  their  business  successfully  and  efficiently  (Tannen,  1990). 

Indirect  methods  of  communication  that  act  to  nurture  relationships  are  often  viewed  as  socializing 
and  therefore  those  individuals  are  perceived  as  not  doing  their  job  or  as  not  taking  their  job  seriously  by 
people  who  use  the  direct  method.  An  example  of  communication  methods  creating  incorrect  assumptions 
about  another  individual  involves  Laura,  a  secondary  media  specialist  and  Pete.  I  observed  Laura  giving  a 
tour  to  visitors  from  another  school  district.  She  showed  them  around  the  media  center  at  the  school  she 
was  employed  at,  pointed  out  the  various  technologies  available  and  the  reasons  the  district  supported  those 
technologies  in  the  schools.  Laura  addressed  their  questions  regarding  all  aspects  of  technology  used  and 
those  she  could  not  answer  encouraged  them  to  call  and  talk  with  Pete.  She  was  an  excellent  ambassador 
for  the  district  and  for  the  use  of  technology  in  school  media  centers.  However,  Pete  perceives  Laura  as  a 
socialite  and  as  someone  who  "just  sits  in  her  office  waiting  to  retire." 

Observations 

A  media  director  should  be  an  administrative  position  who  orchestrates  the  use  of  technology 
within  the  school  district.  The  duties  and  responsibilities  of  the  media  director  evolve  as  the  district's 
climate  and  resources  do.  Regarding  his  own  definition  of  his  position,  Pete  fluctuates.  For  example, 
when  asked  what  it  is  he  does,  the  first  descriptor  Pete  uses  is  facilitator,  yet,  he  is  frustrated  by  the 
secondary'  media  specialists  for  viewing  him  in  this  light. 

Pete  is  a  leader,  promoted  from  the  ranks  of  media  specialists.  However,  his  administrative  duties 
have  caused  him  some  "fellowship"  difficulties  and  he  has  opted  to  "wait  them  [media  specialists]  out."  A 
leader's  legitimacy  is  defined  by  the  followers'  acceptance  of  him  as  a  leader  and  of  him  as  a  part  of  the 
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group.  As  Pete  waits  for  his  position  to  become  legitimate,  he  becomes  less  a  member  of  the  group  he 
leads. 

As  director,  Pete  must  have  effective  communication  skills.  However  without  conscientiously 
realizing  it,  gender  has  played  a  part  in  his  attitude  and  dealings  with  several  of  the  media  specialists.  Pete 
uses  the  direct  method  of  communication,  which  is  typical  of  men.  Most  of  the  medial  specialists  used  the 
indirect  approach,  which  is  typical  of  women.  Pele's  perceptions  of  many  of  the  media  specialists  as 
"socialites"  and  "time  wasters"  are  a  direct  result  of  the  two  modes  of  communications. 
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Members  of  minority  groups  come  to  college,  especially  community  colleges,  with  varying  levels 
of  computer  experience,  but  generally  speaking,  their  experience  is  not  as  extensive  as  that  brought  by  non- 
minority  students.  This  initial  imbalance  has  both  immediate  and  long-range  effects  on  minority  students' 
academic  success  and  their  eventual  career  opportunities.  However,  institutions  of  higher  education, 
especially  community  colleges,  can  develop  programs  to  address  these  initial  inequities.  This  paper  reviews 
the  research  on  the  effects  of  differential  computer  background  on  the  short-  and  long-range  success  of 
minority  students  identifies  some  strategies  Glendale  Community  College  has  used  to  encourage  minority 
students'  use  of  computing,  specifically  computer  conferencing,  and  explains  the  measures  constructed  to 
track  institutional  progress  in  providing  equal  access. 

Background 

Researchers  have  identified  a  number  of  social  and  economic  reasons  why  minority  students  leave 
the  secondary  schools  with  less  computer  experience  than  non-minority  students.  Issues  of  access  dominate 
(Neuman,  1991 ).  Students  build  their  expertise  by  spending  regular  and  extensive  time  op.  computers,  so 
the  ratio  of  students  to  computers  is  critical.  In  addition,  they  develop  a  sense  of  self-efficacy  by  succeeding 
in  computing  tasks  and  watching  others  like  themselves  succeeding  also  (Olivier  &  Shapiro.  1993).  While 
many  schools  have  inadequate  numbers  of  classroom  computers,  minority  students  are  less  likely  to  have 
alternative  access  points,  either  at  home  or  in  public  libraries  where  they  can  supplement  the  computer  time 
provided  at  school  (Resta,  1992).  In  schools,  more  aggressive  students  are  likely  to  monopolize  computer 
use;  they  frequently  have  access  to  more  sophisticated  programs  and  more  extensive  resources  like  the 
Internet.  Conversely,  minority  students,  because  they  are  often  at-risk,  are  frequently  directed  towards  the 
use  of  "explicit-goal"  software,  the  use  of  which,  according  to  Stanley  Pogrow,  causes  the  learning  gap 
between  them  and  other  students  to  widen  (1993). 

A  number  of  cultural  issues  affect  minority  students'  ability  to  develop  extensive  computer 
expertise.  Often  instruction  in  computing  is  linked  to  math  and  science  instruction;  if  minority  students 
proceed  more  slowly  in  these  content  areas,  they  will  also  proceed  more  slowly  in  developing  computer 
skills  (Martin  &  Hearne,  1989;  Olivier  &  Shapiro,  1993).  In  fact,  the  kinds  of  computing  activities 
available  in  the  past  for  students  have  fallen  into  a  restricted  range,  one  tied  more  closely  to  majority  culture 
than  to  the  interests  of  members  of  minority  groups  (Lucas  &  Schecter.  1992).  In  addition,  in  most 
computer  labs,  few  mentors  and  role  models  exist  for  minority  students.  Finally,  many  teachers  view 
computing  hierarchically,  reserving  it  for  those  students  who  have  mastered  other  skills  perceived  as  more 
"basic"  (Yellin  &  Koetting,  1988).  Consequently,  minority  students,  over  their  secondary  school  careers, 
may  not  develop  equivalent  levels  of  computer  competence  as  their  non-minority  peers. 

When  minority  students  enter  college  with  less  computer  experience,  they  can  feel  the  effects 
almost  immediately.  Any  perceived  inadequacy,  whether  it  is  in  preparation,  experience,  or  skill  level, 
contributes  to  the  ease  with  which  students  are  willing  to  drop  out.  But  differing  levels  of  computer 
experience  have  measurable  effects  on  students'  ability  to  complete  assigned  work  in  a  timely  and  efficient 
manner  (Resta,  1992).  Students  with  well-developed  computer  skills  know  how  to  use  various  productivity 
tools  like  spreadsheets,  databases,  and  word  processors,  and  recognize  their  benefits.  In  addition,  they  may 
know  how  to  use  the  Internet  to  retrieve  up-to-date  or  obscure  information.  Students  who  neither  have 
access  to  the  tools  nor  have  prior  experience  using  them  may  spend  their  available  time  doing  mechanical 
tasks  manually,  running  out  of  time  (and  energy)  before  they  get  to  more  intellectually  rewarding 
activities.  Lack  of  computer  experience,  especially  with  networked  resources,  may  close  off  opportunities 
for  developing  improved  literacy  skills,  and  lack  of  familiarity  with  computer  environments  may  hamper 
minority  students  when  they  encounter  technologically-rich  educational  activities  like  simulations  (Yellin 
&  Koetting.  1988).  In  addition  to  inhibiting  students'  academic  success,  lack  of  computer  experience  can 
close  off  opportunities  for  career  development,  and  thus  restrict  minority  students'  chances  of  long-term 
economic  success  (Badagliacco,  1990). 
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Local  Strategies 


All  institutions  of  higher  education,  but  especially  community  colleges,  have  responsibility  for 
fostering  increased  educational  access  and  for  ensuring  opportunities  for  success  for  minority  students.  In 
part,  they  can  do  this  by  encouraging  students  to  increase  their  level  of  computer  expertise,  both  before  and 
after  they  come  to  college.  Glendale  Community  College  has  provided  extensive  computing  facilities  and 
a  variety  of  user  environments  so  that  members  of  minority  groups  can  gain  easy  access  and  learn  in  a 
hospitable  climate.  However,  perceived  availability  of  resources  may  not  translate  into  actual  use. 
Second,  computing  activities  have  been  linked  with  activities  across  instructional  levels  and  throughout  the 
curriculum,  so  that  lack  of  progress  in  one  academic  area  does  not  preclude  the  development  of  computing 
expertise.  In  addition,  this  campus  has  provided  different  kinds  of  computer-mediated  environments  in  order 
to  tap  different  kinds  of  learning  skills  demonstrated  by  students  with  a  variety  of  cultural  backgrounds.  It 
has  provided  a  variety  of  computer-mediated  communication  opportunities,  especially  ones  that  take  into 
account  backgrounds  in  oral  culture,  where  students  have  been  able  to  practice  their  own  literacy  skills  by 
reading  and  writing  on  topics  that  relate  to  their  own  interests  and  heritage.  Finally,  GCC  has  placed 
minority  teachers,  tutors,  and  assistants  in  open  computer  environments  where  they  are  visible  to  minority 
students  and  can  model  computer  competence.  Investigation  into  the  impact  of  these  strategies  is  ongoing 
and  involves  a  number  of  research  methods,  both  quantitative  and  qualitative. 


Method 

This  research  analyzes  various  aspects  of  the  computer  conferencing  activity  of  nearly  12,000 
community  college  students  over  a  two-and-a-half-year  period  at  Glendale  Community  College,  a  campus  of 
the  Maricopa  Community  College  District.  These  students  contributed  to  over  250  computer  conferences 
aligned  with  conventional  classroom  instruction  across  the  disciplines,  including  classes  in  mathematics, 
science,  social  science,  and  humanities,  as  well  as  all  levels  of  writing  courses.  The  conferencing  system, 
called  the  Electronic  Forum  (EF),  supports  extensive  record-keeping  (time  in  conference,  number  of  entries 
read,  number  of  entries  written,  number  of  words  written)  and  imports  demographic  information  from  the 
Student  Information  System  (gender,  ethnicity,  age,  and  current  course  load).  Although  the  bulk  of  this 
research  uses  data  only  from  course-related  discussions,  the  conferencing  program  is  used  heavily  by 
students  for  both  course-related  and  personal  communication.  At  this  campus,  where  this  conferencing 
program  is  most  fully  developed,  class  discussions  represent  about  10%  of  the  total  on-line  activity. 
During  the  academic  year  1 992-93,  approximately  5,000  students  exchanged  1 .2  million  pieces  of  private 
mail,  and  during  fall  semester  of  1993,  nearly  2400  students  spent  an  average  of  2  hours  each  week  reading 
and  writing  online. 

Research  Questions 

The  volume  of  writing  activity  demonstrated  by  students,  and  the  potential  it  implies  for  literacy 
skills  development,  has  prompted  a  number  of  questions  about  equity:  is  participation  in  this  program 
accessible  to  students  equally?  That  is,  do  the  students  who  use  the  Electronic  Forum  reflect  the 
demographic  make-up  of  the  student  population  as  a  whole? 

Findings 

Analysis  of  data  collected  from  class-based  conferences  over  five  semesters  reveals  that  minority 
enrollment  in  classes  incorporating  electronic  conferences  seems  to  match  or  exceed  general  enrollment 
demographics  of  the  community  college  with  respect  to  ethnicity.  That  is,  a  pattern  of  restricted  access  by 
minority  students  to  this  computer  conferencing  system  does  not  seem  to  be  emerging.  In  fact,  in  four  of 
the  five  semesters,  minority  students  (Native  Americans,  Asians,  Blacks,  and  Hispanics)  are  over- 
represented  among  those  students  who  have  access  to  EF  as  part  of  their  required  classwork,  although  in  any 
given  semester,  EF  users  represent  only  between  12-17%  of  the  student  population  on  campus.  Tables  1-5 
illustrate  the  patterns  for  various  groups  for  each  semester;  with  the  exception  of  Spring,  1992  when  there 
was  no  significant  difference  in  the  access  of  various  groups  (p=.1012),  during  the  following  four  semesters, 
minority  students  were  significantly  over-represented  among  EF  users  (p<.0000). 
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Observed 

Expected 

Residual 

Native  American 

21 

14.03 

6.97 

Asian 

35 

32.23 

2.77 

Black 

40 

31.30 

8.70 

Hispanic 

124 

114.73 

9.27 

White,  non  Hispanic 

841 

868.71 

-27.71 

Chi-square=7.7499  D.F.=4  p=.1012 


Table  1.    College  Demographics  compared  with  EF  Demographics 

(Spring  Semester,  1992) 


Observed 

Expected 

Residual 

Native  American 

29 

19.51 

9.49 

Asian 

51 

48.36 

2.64 

Black 

74 

44.02 

29.98 

Hispanic 

200 

162.00 

38.00 

White,  non  Hispanic 

1028 

1 108.10 

-80.10 

Chi-square=39.8719  D.F.=4  p=.0000 


Table  2.    College  Demographics  compared  with  EF  Demographics 

(Fall  Semester,  1992) 


Observed 

Expected 

Residual 

Native  American 

28 

24.20 

3.80 

Asian 

80 

54.09 

25.91 

Black 

73 

51.11 

21.89 

Hispanic 

247 

176.96 

70.04 

White,  non  Hispanic 

1194 

1315.64 

-121.64 

Chi-square=76 1.3558  D.F.=4  p=.0000 


Table  3.     College  Demographics  compared  with  EF  Demographics 

(Spring  Semester,  1993) 


Observed 

Expected 

Residual 

Native  American 

30 

21.62 

8.38 

Asian 

57 

50.55 

6.45 

Black 

52 

40.47 

11.53 

Hispanic 

230 

159.05 

70.95 

White,  non  Hispanic 

1026 

1123.3! 

-97.31 

Chi-square=47.4417  D.F.=4  p=.0000 


Table  4.    College  Demographics  compared  with  EF  Demographics 
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(Fall  Semester,  1993) 


Observed 

Expected 

Residual 

Native  American 

31 

27.17 

3.83 

Asian 

69 

59.91 

9.09 

Black 

1 14 

54.94 

59.06 

Hispanic 

276 

189.57 

86.43 

White,  non  Hispanic 

1150 

1308.41 

-158.41 

Chi-square=l  23.9853  D.F.=4  p=.(XX)0 


Table  5.     College  Demographics  compared  with  EF  Demographics 

(Spring  Semester,  1994) 

Approximately  77%  of  those  students  with  EF  access  for  their  classes  actually  participate; 
however,  no  significant  difference  in  the  ethnicity  of  participants  as  compared  with  those  merely  having 
access  has  appeared  (p<.0O0O).  Tables  6-10  show  the  demographic  makeup  of  EF  users  compared  with  the 
demographic  makeup  of  the  EF  population  as  a  whole  for  each  of  the  semesters  studied.  Non-participants 
are  students  who  neither  read  nor  wrote  in  the  class  conference  for  the  entire  semester;  some  of  them  may 
have  dropped  the  course. 


Observed 

Expected 

Residual 

Native  American 

15 

16.55 

-1.55 

Asian 

26 

27.58 

-1.58 

Black 

29 

31.52 

-2.52 

Hispanic 

93 

97.70 

-4.70 

White,  non  Hispanic 

673 

662.65 

10.35 

Chi-square=.8239  D.F.=4  p=.9352 

Table  6.    EF  User  Demographics  compared  with  General  EF  Demographics 

(Spring  Semester,  1992) 


Observed 

Expected 

Residual 

Native  American 

21 

23.57 

-2.57 

Asian 

38 

41.44 

-3.44 

Black 

60 

60.13 

-.13 

Hispanic 

153 

162.52 

-9.52 

White,  non  Hispanic 

851 

835.34 

15.66 

Chi-square=1.4163  D.F.=4  p=.8414 

Table  7.    EF  User  Demographics  compared  with  General  EF  Demographics 

(Fall  Semester,  1992) 


Observed 

Expected 

Residual 

Native  American 

19 

21.13 

-2.13 

Asian 

54 

60.37 

-6.37 

Black 

54 

55.09 

-1.09 

Hispanic 

181 

186.39 

-5.39 

White,  non  Hispanic 

916 

901.02 

14.98 

Chi-square=1.3132  D.F.=4  p=.8591 


Table  8.    EF  User  Demographics  compared  with  General  EF  Demographics 
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(Spring  Semester,  1993) 


Observed 

Expected 

Residual 

Native  American 

15 

23.72 

-8.72 

Asian 

43 

45.07 

-2.07 

Black 

34 

41.12 

-7.12 

Hispanic 

181 

181.86 

-.86 

White,  non  Hispanic 

830 

811.24 

18.76 

Chi-square=4.9702  D.F.=4  p=.2904 

Table  9.    EF  User  Demographics  compared  with  General  EF  Demographics 

(Fall  Semester,  1993) 


Observed 

Expected 

Residual 

Native  American 

22 

22.95 

-.95 

Asian 

42 

51.08 

-9.08 

Black 

79 

84.39 

-5.39 

Hispanic 

195 

204.31 

-9.31 

White,  non  Hispanic 

876 

851.28 

24.72 

Chi-square=3.1380  D.F.=4  p=.5350 


Table  10.    EF  User  Demographics  compared  with  General  EF  Demographics 

(Spring  Semester,  1994) 


Specific  findings  with  respect  to  particular  ethnic  groups  may  be  of  interest: 

1 .  Native  American  students  represent  about  1.5%  of  GCC's  student  population  and  are  consistently  over- 
represented  among  students  with  EF  access.  Their  access  has  increased  slightly  over  the  past  five  semesters. 
On  the  average,  33%  of  the  Native  American  students  chose  neither  to  read  nor  write  in  their  class 
conferences. 

2.  Asian  students  generally  represent  about  3%  of  GCC's  student  population  and  are  slightly  over- 
represented  among  students  with  access  to  EF.  Their  access  has  fluctuated  over  the  past  five  semesters. 
On  the  average,  30%  of  the  Asian  students  chose  neither  to  read  nor  write  in  their  class  conferences. 

3.  Black  students  represent  slightly  less  than  3%  of  GCC's  student  population  and  are  over-represented 
among  students  who  have  access  to  EF.  Their  access  has  generally  increased  over  the  past  five  semesters. 
On  the  average,  27%  of  Black  students  chose  neither  to  read  nor  write  in  their  class  conferences. 

4.  Hispanic  students  represent  nearly  1 1  %  of  GCC's  population  and  are  over-represented  among  students 
using  EF  for  classes.  Their  access  has  generally  increased  over  the  past  five  semesters.  On  the  average, 
25%  of  Hispanic  students  chose  neither  to  read  nor  write  in  their  class  conferences. 

5.  White  students  generally  account  for  75-80%  of  GCC's  student  population  and  are  generally 
underrepresented  among  EF  users.  Their  access  has  generally  increased  over  the  past  five  semesters.  On 
the  average,  21%  of  White  students  chose  neither  to  read  nor  write  in  their  class  conferences. 

While  these  figures  about  access  and  overall  use  are  encouraging,  some  statistically  significant 
differences  have  appeared  with  respect  to  the  volume  of  participation  of  members  of  different  ethnic  groups 
in  class  conferences.  It  is  becoming  clear  that  members  of  some  groups  willingly  participate  and  others 
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write  less  frequently  and  at  less  length.  This  may  represent  actual  inequities  in  literacy  instruction  (Lucas 
&  Schecter,  1992),  subtle  differences  in  communication  environments  (Willis,  1991),  or  the  institution's 
failure  to  be  proactive  in  addressing  diversity  (Trueba,  19)3).  These  differences  have  not  been  consistent 
across  semesters  nor  across  groups,  and  their  analysis  is  made  even  more  difficult  because  of  the  occasional 
presence  of  some  very  active  writers.  In  addition,  gender  may  be  a  contributing  factor.  For  example, 
preliminary  analysis  suggests  that  black  women  are  more  comfortable  in  class  conferences  (as  measured  by 
the  amount  of  writing  they  do)  than  black  men.  However,  we  can  now  develop  baseline  data  for  extensive 
longitudinal  studies  which  can  identify  differences  in  participation  and  look  for  strategies  to  address  them. 

Alternative  Learning  Environments 

Several  researchers  have  suggested  that  in  order  to  address  the  needs  of  minority  students 
effectively,  educational  institutions  will  have  to  develop  alternatives  to  the  conventional  pattern  of  lecture 
and  large-group  discussion  (Light,  1990).  Emerging  data  with  respect  to  the  participation  of  minority 
students  in  voluntary  on-line  discussions  such  as  IRC  suggests  that  different  kinds  of  environments, 
especially  ones  that  seem  more  "oral,"  may  appear  more  hospitable  to  some  students,  encourage  greater 
participation,  and  spark  an  important  kind  of  meta-cognition  that  students  can  avoid  in  other  kinds  of 
learning  environments  (Shedletsky,  1993;  Pogrow,  1993).  In  an  attempt  to  find  or  create  these  spaces, 
DeVaney  (1993)  suggests  that  members  of  minority  groups  should  be  able  to  construct  technological 
environments  that  reflect  their  culture  and  practices. 

Many  features  of  computer-mediated  computer  environments  can  affect  student  interest,  comfort, 
and  learning,  so  identifying  specific  qualities  will  be  quite  difficult.    In  an  attempt  to  discover  which 
groups  find  which  environments  more  engaging  (and  following  up  on  some  personal  observations  of 
computer  activity),  we  analyzed  the  number  of  students  participating  in  one  such  environment,  Internet 
Relay  Chat  (IRC),  for  the  first  fifteen  weeks  of  the  Spring,  1994  semester  and  compared  that  distribution 
with  the  campus  population  as  a  whole.  Asian  students  were  significantly  over  represented  in  this  group. 


Observed 

Expected 

Residual 

Native  American 

12 

11.88 

.12 

Asian 

67 

26.19 

40.81 

Black 

18 

24.02 

-6.02 

Hispanic 

71 

82.88 

-1 1.88 

White,  non  Hispanic 

549 

572.03 

-23.03 

Chi-square=67.7158         D.F.=4  p=.0000 

Table  13.    Demographic  Analysis  of  Internet  Relay  Chat  Participants  compared  with 

GCC  Enrollment 

(Spring  Semester,  1994) 


Compared  with  th\  general  demographic  makeup  of  EF  users,  both  Asian  and  White  students  were  over- 
represented  among  IRC  participants. 


Observed 

Expected 

Residual 

Native  American 

12 

13.55 

-1.55 

Asian 

67 

30.17 

36.83 

Black 

18 

49.84 

-31.84 

Hispanic 

71 

120.67 

-49.67 

White,  non  Hispanic 

549 

502.77 

46.23 

Chi-square=90.1855         D.F.=4  p=.0O0O 

Table  14.    Demographic  Analysis  of  Internet  Relay  Chat  Participants  compared  with 

EF  User  Demographics 

(Spring  Semester,  1994) 


There  are  a  number  of  reasons  why  Asian  students  might  find  IRC  more  interesting  than  other 
forms  of  online  communication,  not  the  least  of  which  may  be  their  ability  to  communicate  on  various 
channels  in  their  native  languages  with  other  Asian  students  in  the  United  States  or  around  the  world. 
Then  again,  the  interest  may  be  in  the  synchronicity  of  communication,  the  speed  and  immediacy  of 
response.  In  addition  to  surveying  students  who  are  active  IRC  users  and  analyzing  their  responses,  a 
strategy  possible  with  computer-mediated  communication  (Beals,  1992),  our  next  step  will  be  to  bring  in  a 
number  of  Internet  discussion  lists  that  might  be  of  interest  to  Asian  students,  some  in  languages  other 
than  English.  By  analyzing  student  participation  in  asynchronous  discussions,  we  may  be  able  to  determine 
whether  the  attraction  is  cultural  or  structural. 

Providing  Role  Models 

Students'  sense  of  their  own  computing  competence  is  affected  not  only  by  their  own  ability  to 
perform  adequately,  but  also  by  their  observations  of  others  similar  to  themselves  who  are  also  successful 
(Olivier  &  Shapiro,  1993).   Role  models  for  minority  students  are  critical  (Resta,  1992);  that  means  that 
not  only  must  there  be  adequate  numbers  of  minority  group  members  who  can  model  computer  competence, 
but  they  must  also  be  visible,  interacting  with  students  on  a  regular  basis.  The  over-representation  of 
minority  students  among  EF  users  is  encouraging  in  this  respect. 

Architecture  and  staffing  policies  can  make  a  difference  in  the  "visibility"  of  role  models. 
Glendale  has  two  large,  open  computing  facilities  which  make  use  of  student  workers  who  have 
demonstrated  computing  expertise.  More  so  even  than  faculty  members,  these  individuals  work  closely 
with  students,  providing  motivation  and  assistance,  modeling  computer  competence.  The  demographic 
makeup  of  this  group  during  Spring,  1994,  reflects  the  diversity  of  the  campus  with  respect  to  ethnicity 
except  for  participation  of  Native  Americans.  The  difference  with  respect  to  gender  between  Instructional 
Associates  and  college  enrollment  is  non-significant,  although  women  are  under-represented. 


Observed 

Expected 

Residual 

Native  American 

0 

Asian 

4 

3.12 

.88 

Black 

3 

2.86 

.14 

Hispanic 

13 

9.87 

3.13 

White,  non  Hispanic 

64 

68.15 

-4.15 

Chi-square=1.4972  D.F.=3  p=.6829 

[Chi-square  statistic  questionable  because  2  cells  have  expected  frequencies  of  less  than  5;  minimum  cell 

expected  frequency  is  2.9] 


Observed 

Expected 

Residual 

Male  IAs 

45 

38.39 

6.61 

Female  IAs 

40 

46.61 

-6.61 

Chi-square=2.0735  D.F.=1  p=1499 


Table  15.    Demographic  Analysis  of  Instructional  Associates  as  compared  with 

College  Enrollment 

(Spring  Semester,  1994) 

Recommendations 

In  many  cases,  providing  technological  environments  that  foster  increased  access  to  learning  for 
members  of  minority  groups  can  be  accomplished  at  little  or  no  cost  (Martin  &  Hearne,  1989). 
Obviously,  it  is  a  useful  barometer  of  access  to  continue  to  collect  data  each  semester  with  respect  to  the 
participation  of  minority  group  members  in  discussions  aligned  wi;h  their  classes  and  in  the  diversity  of  the 
group  of  Instructional  Associates  hired  to  help  those  students.  Cross-district  analysis  (and  cross- 
institutional  analysis  as  well)  may  help  us  identify  policies,  climates,  and  strategies  that  are  effective  (or 
not,  as  the  case  may  be).  In  addition,  the  following  initiatives  can  be  launched  at  little  or  no  cost: 
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•  Increase  the  variety  of  courses  (and  the  range  of  instructional  levels)  that  employ  computer- 
mediated  communication  as  an  instructional  tool; 

•  Identify  (by  textual  analysis  of  class  discussions  and  surveys)  issues  of  interest  and  importance 
to  minority  students; 

•  Sensitize  teachers  to  different  kinds  of  communication  practices  common  among  different  ethnic 
groups  and  show  how  they  are  revealed  in  online  communication; 

•  Work  with  campus  clubs  and  groups  as  "sites"  of  peer-based  learning; 

•  Develop  connections  with  the  feeder  high  schools  to  introduce  college-bound  minority  students 
to  a  variety  of  open  computing  environments; 

•  Enlist  members  of  minority  groups  to  "scour"  the  Internet  for  additional  resources  of  particular 
interest; 

•  Encourage  teachers  and  students  to  collaborate  on  creating  unconventional  learning 
environments. 

Enhancing  minority  students'  facility  with  technology,  especially  in  ways  that  recognize  and  respect  ethnic 
differences,  may  have  immediate  and  long-range  effects  on  students'  persistence,  academic  success  and  career 
satisfaction. 
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teaching  of  reading  and  writing.  Urban  Review.  24(2).  85-104. 

Literacy  instruction  for  minority  students  is  affected  by  the  characteristics  of  the  individual  students,  by 
sociocultural  elements,  by  language  differences  and  by  instructional  culture.  It  is  the  responsibility  of 
schools  to  redress  language  instruction  inequities. 

Martin,  B.,  &  Hearne,  J.  D.  (1989).  Computer  equity  in  education.  Educational  Technology. 
29(11),  47-51. 

Minority  group  members,  women,  and  disabled  persons  frequently  lack  computer  experience;  this  hurts  their 
career  development  and  helps  to  reinforce  the  stereotypical  notion  that  they  have  less  computer  aptitude. 
Cultural  and  economic  forces  help  perpetuate  this  differential  development  of  computer  experience,  but  most 
of  the  variation  in  computer  access  for  different  groups  can  be  redressed  by  administrative  leadership  and 
personnel  commitment.  Enhanced  consciousness  can  change  these  patterns. 

Neuman,  D.  (1992).  Technology  and  equity.  D.  P.  Ely,  &  B.  B.  Minor  (editor),  Educational  Media 
and  Technology  Yearbook  Vol.  18,  (pp.  100-103).  Englewood,  CO:  Libraries  Unlimited. 

Using  technology  does  not  automatically  promote  equity.  The  ratio  of  students  to  computers  is  highest  in 
wealthiest  districts  and  lowest  in  the  poorer  districts.  In  addition,  it  is  common  for  students  at  the  lowest 
levels  to  be  relegated  to  using  the  least  imaginative  software.  In  some  schools,  computers  are  located  in 
areas  to  which  only  the  most  advanced  students  have  access.  Finally,  much  educational  software  makes  use 
of  sounds,  graphics,  and  plots  that  reinforce  "masculine"  traits  and  thus  encourage  girls  in  their  negative 
attitudes. 

Olivier,  T.  A.,  &  Shapiro,  F.  (1993).  Self-efficacy  and  computers.  Journal  of  Computer-based 
Instruction.  20(3),  81-85. 

Self-efficacy,  the  perception  of  one's  ability  to  perform  specific  tasks,  is  a  judgment  derived  from  success 
in  one's  efforts,  observations  of  others'  succeeding,  encouragement  from  others,  and  anxiety.  Behavior 
modeling,  because  it  enhances  one's  sense  of  self-efficacy,  is  a  more  effective  teaching  strategy  than  use  of 
computer  tutorials.  Self-efficacy  with  respect  to  computers  is  directly  correlated  with  mathematics 
confidence  and  perceived  mathematics  ability.  Various  assessment  instruments  exist  to  measure  self- 
efficacy,  and  their  use  could  point  to  opportunities  for  strategic  interventions  and  instructional 
improvement. 

Padilla,  R.  V.  (1992).  Using  dialogic  research  methods  to  study  Chicano  college  students.  Urban 
Review,  24(3),  175-183. 

Dialogic  research  methods,  Freire's  "psycho-social  method,"  can  help  researchers  discover  "generative 
themes"  embedded  in  a  particular  group  of  students.  This  research  identified  three  themes:  a  sense  of  social 
isolation  from  family,  a  recognition  of  the  personal  cost  of  academic  achievement,  and  a  feeling  of 
ambiguity  about  ethnic  identity.  These  themes  were  worked  out  in  both  inter-ethnic  and  intra-ethnic 
struggles.  Identification  of  these  conflicts  can  point  to  intervention  strategies  that  will  help  Hispanic 
students  persist  in  their  education. 
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Pogrow,  S.  (1993).  A  learning  drama  approach  to  using  computers  with  at-risk  students.  R. 
Muffoletto,  &  N.  N.  Knupfer  (eds.),  Computers  in  education:  social,  political  &  historical  perspectives, 
(pp.  207-217).  Cresskill,  NJ:  Hampton  Press. 

At-risk  students  (beyond  the  third  grade)  do  not  benefit  from,  and  may  actually  be  harmed  by  the  use  of 
explicit-goal  software  (conventional  CAI  as  well  as  word-processing  programs  and  simulations).  Such 
software  assumes  that  at-risk  students  fail  to  learn  because  they  do  not  practice  working  with  concepts,  but 
current  research  shows  that  the  problem  is  more  a  failure  to  recognize  and  apply  mental  strategies  to 
problems.  That  is,  at-risk  students  do  not  know  what  it  means  to  understand;  they  do  not  know  how  and  in 
what  situations  to  generalize,  hypothesize,  or  predict.  Better  software  provides  intriguing  problem-solving 
settings  that  spawn  conversations  about  what  is  happening,  encouraging  students  to  develop  critical 
thinking  skills. 

Resta.  P.  (1992).  Organizing  education  for  minorities:  enhancing  minority  access  and  use  of  the 
new  information  technologies  in  higher  education.  Education  &  Computing.  8.  1 19-127. 

Minority  students,  who  generally  enter  college  with  less  background  and  experience  with  computing,  do  not 
have  equal  access  to  tools  which  help  majority  students  achieve  academically,  and  thus  can  remain 
"information-disadvantaged."  Providing  special  computing  programs  directed  toward  minority  students, 
encouraging  minority  students  to  develop  their  computer  skills,  recruiting  minority  students  as  workers  in 
computing  labs,  and  building  cohesive  minority  groups  through  electronic  networks  are  strategies  that 
institutions  of  higher  education  can  use  to  redress  these  imbalances. 

Ross,  S.  M.,  Smith,  L.,  Morrison,  G.,  &  Erickson,  A.  (1989).  An  apple  a  day  and  at  night:  A 
distance  tutoring  program  for  at-risk  students.  Educational  Technology.  22(8),  23-28. 

Distance  tutoring,  in  which  at-risk  students  have  increased  opportunities  to  interact  individually  with 
teachers,  can  make  use  of  the  increased  educational  benefits  of  personalized  instruction  and  mitigate  the 
logistical  problems  of  time  and  expense.    This  project  had  beneficial  effects  on  student  writing. 

Shedletsky,  L.  (1993).  Minding  computer-mediated  communication:  CMC  as  experiential  learning. 
Educational  Technology.  33(12),  5-10. 

Computer-mediated  communication  can  promote  four  different  kinds  of  ler. -ning:  response  learning 
(negotiating  commands  and  keystrokes),  situation  learning  (developing  ideas  about  the  meaning  and 
usefulness  of  CMC),  transsituation  learning  (reflecting  on  changes  in  one's  interpretation  of  the  CMC 
experience),  and  transcendent  learning  (inventing  new  ways  of  thinking  about  CMC). 

Trueba.  H.  T.  (1993).  Race  and  ethnicity:  The  role  of  universities  in  healing  multicultural 
America.  Educational  Theory.  43(  I ),  4 1  -54. 

Educational  institutions  can  heal  bitter  social,  political,  and  economic  divisions  between  different  ethnic 
groups  by  promoting  a  "multicultural  literacy"  in  which  cultural  differences  are  acknowledged  and  where 
new  means  of  communication  across  them  are  practiced.   Schools  are  not  neutral  institutions;  they  are 
political  environments  in  which  a  social  order  (usually  the  current  one)  is  perpetuated.  Given  the  current 
crisis,  schools  must  become  more  active  in  addressing  issues  of  educational  inequity  presented  by  the 
increasing  proportion  of  immigrants,  refugees  and  minorities  in  society. 

Willis,  J.  (1991).  Computer  mediated  communication  systems  and  intellectual  teainwork:  Social 
psychological  issues  in  design  and  implementation.  Educational  Technology.  21(4).  10-20. 

Researchers  usually  identify  either  engineering  and  technical  reasons  or  business  and  marketing  reasons  for 
the  success  or  failure  of  technological  innovations;  however,  the  social  and  behavioral  sciences  can  also 
provide  analytic  frameworks.  Electronic  communication  can  provide  access  to  information  (either  "standard" 
or  "specialized"),  can  enable  participants  to  exchange  information,  or  can  enable  collaboration  (either  to 
solve  problems  or  to  develop  and  exchange  information).  These  different  uses  may  require  tailored 
communication  systems. 
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Yellin,  D.,  &  Koetting,  J.  R.  (1988).  Literacy  instruction  and  children  raised  in  poverty:  A 
theoretical  discussion.  Journal  of  Curriculum  Theorizing.  £,  101-1 14. 


In  order  to  improve  the  educational  success  of  disadvantaged  students,  schools  must  employ  a  new  form  of 
literacy  instruction  that  focuses  on  literacy  as  a  means  for  the  communication  of  ideas.    The  history  of 
both  reading  and  writing  instruction  has  followed  a  circular  route:  beginning  with  instruction  in  isolated 
skills,  moving  to  embedded  instruction,  and  then  back  to  emphasis  on  teaching  discrete  or  so-called  "basic" 
skills.  Current  educational  practice  with  its  emphasis  on  pretest-posttest  cycle,  standardized  curriculum, 
specified  outcomes,  and  measurable  objectives  has  served  to  reinforce  this  reductive,  "techno-rational" 
picture  of  literacy.  An  alternative  model  promotes  "real  reading  and  writing"  of  texts  that  have  personal 
importance  to  students'  lives. 
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Abstract 


The  effectiveness  of  an  evaluation  model  for  interactive  television  courses  was  studied.  Inquiry  was 
conducted  with  distance  education  graduate  students  to  obtain  their  authentic  voices  about  a  traditional 
model:  Small  Group  Instructional  Diagnosis  (SGID),  a  facilitator-based  model.  Analysis  of  students'  on-line 
comments  and  administrative  actions  indicated  that  SGID  is  effective  for  these  students.  Student  interaction 
between  the  process  and  facilitator  were  cited  as  important  in  the  helpfulness  of  the  process.  Items  on 
hindering  student  interactions  with  the  process  were  identified  by  more  remote  site  students  than  by  those 
on  campus.  On-site  students  were  more  apt  to  indicate  no  interactions  hindered  the  process  and  that  no 
changes  were  needed.  Administrators  demonstrated  change  activity.  These  results  are  consistent  with  the  data 
on  SGID  and  have  potential  for  those  evaluating  graduate  courses  delivered  by  two-way  audio  and  video. 


Applying  an  Interactive  Evaluation  Model  to  Interactive  Television 

Introduction 

The  use  and  the  effectiveness  of  student  evaluations  concerning  course  offerings  and  instructor 
effectiveness  have  been  documented  in  both  distance  education  and  traditional  classroom  settings. 
Numerous  studies  of  distance  education  incorporate  student  evaluations  as  one  or  more  of  the  tools  used  to 
assess  course  effectiveness  (Dillon,  Gunawardena,  &  Parker,  1992;  Schlosser  &  Anderson,  1994;  Wagner, 
1993).  Maitino's  review  of  current  research  on  student  evaluation  of  college  instruction  in  traditional 
classrooms  (1992)  supports  the  reliability  and  relevancy  of  such  data. 

In  1979,  Clark  and  Bekey  refined  a  University  of  Massachusetts  clinical  model  that  provides  mid- 
semester  evaluative  feedback  to  the  instructor  based  on  oral  information  given  by  small  student  groups  to  an 
external,  trained  facilitator  who  spends  one  class  period  alone  with  the  students.  During  the  period,  after 
separating  the  students  into  small  groups,  the  facilitator  directs  all  the  students  in  each  group  to  provide 
answers  to  two  questions  about  everything  that  helps  and  hinders  their  learning  in  the  course.  A  third 
question  is  also  posed  that  asks  for  their  input  on  any  needed  changes.  After  a  recorder  from  each  group 
provides  all  the  responses,  the  facilitator  leads  the  class  in  clarifying  the  responses  to  assure  mutual 
agreement  on  meanings.  The  class  then  votes  on  each  response,  indicating  the  degree  to  which  each 
student  supports  each  item.  After  the  class  session,  the  facilitator  meets  with  the  instructor,  providing  the 
ranked  results  of  the  student  responses,  interpretative  comments,  and  suggestions  for  improving  instruction. 

This  process,  the  Small  Group  Instructional  Diagnosis  (SGID),  that  has  appeared  in  the  literature 
on  teaching  effectiveness  as  one  type  of  evaluation  technique,  has  been  shown  to  open  the  lines  of 
communication  between  faculty  and  students  and  (Seldin,  1988). 

Recently  there  has  been  an  increased  interest  in  conducting  research  that  encourages  the  application 
of  research  methods  that  produce  results  leading  to  immediate  change  and  supporting  improved  practice  in 
teaching.  While  there  is  debate  over  the  distinction  between  practical  classroom  inquiry  and  "teacher  as 
formal  educational  researcher"  (Richardson,  1994,  p.  7),  a  distinguishing  feature  of  the  latter  is  applicability 
of  results  to  a  broader  community.  Such  qualitative  studies  have  the  potential  of  providing  researchers  with 
opportunities  for  describing  natural  habitats  as  they  are  with  minimal  disturbance;  providing  insights  to 
existing  themes  and  addressing  immediate  needs. 

No  one  source  of  data  is  sufficient  for  evaluating  the  efficacy  of  an  instructional  situation  (Seldin, 
1988).  While  a  concern  facing  those  engaged  in  qualitative  studies  is  structuring  and  validating  the 
resulting  data,  triangulation  is  an  approach  that  lends  support  to  the  analysis.  Through  comparisons  of 
interview  data  with  observational  data  and  anecdotal  reports,  and  by  investigating  participants'  comments 
and  behaviors,  patterns  can  often  be  seen  to  emerge  that  support  hypotheses,  or  lead  to  revised  insights 
(Patton,  1987).  This  study,  while  based  primarily  on  data  gathered  from  student  reports  on  their 
perceptions  of  the  effectiveness  of  a  formalized  evaluation  process,  also  provides  data  from  actions  of  the 
stakeholders:  program  administrators,  faculty  members  and  Department  of  Education  teachers. 

A  structured,  open-ended  questioning  format  familiar  to  the  distance  education  students  was  selected 
to  provide  a  vehicle  in  which  their  authentic,  diverse  voices  could  be  heard  as  they  provided  their 
perceptions  of  learning  in  two  distance  education  courses. 
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In  the  body  of  work  on  distance  education,  the  emphasis  on  evaluating  the  impact  of  distance  education 
courses  can  be  categorized  as: 

Comprehensive:  The  overall  approach  and  design  of  evaluation  of  distance  education  (Thorpe,  1988; 

Zvacek,  1994). 

Administrative:  The  uses  of  distance  education  (Dillon,  Gunawardena,  &  Parker,  1992)  and  its  costs 
(Willey,  1991;  Zvacek,  1994). 

Instructional  Design:  The  study  of  student  learning  in  terms  of  suitability  of  courses  (Willey,  1991) 
and  student  perceptions  of  interaction  (Fulford  &  Zhang,  1994). 

•  Learner  Achievement  and  Attitude:  The  factors  contributing  to  student  success  and  positive  attitudes  in 
distance  education  situations  (Dillon,  Gunawardena,  &  Parker,  1992;  Lauzon,  1992;  Schlosser  & 
Anderson  1994;  Verduin  &  Clark,  1992;  Walker  &  Hackman  1992;  Willis,  1991). 

•  Technical:  The  effectiveness  of  distance  education  courses  in  terms  of  delivery  systems  (Egan,  Welch, 
Page  &  Sebastian,  1992;  Schlosser  &  Anderson,  1994). 

Among  the  emerging  trends  in  distance  education  (Birenbaum,  Glick,  &  Kinsler,  1994),  is  the 
continued  emphasis  on  examining  the  effects  of  distance  education  on  the  learner.  This  study  used  such  a 
focus —  with  the  focus  on  learners  who  were  involved  in  a  new  cooperative  distance  education  university 
and  school  department  partnership.  By  employing  a  broadly  structured,  open-ended  format  for  obtaining  data 
from  the  learners,  this  study  allowed  student  input  in  any  of  the  areas  for  evaluating  distance  education: 
comprehensive,  administrative,  instructional  design,  learner  achievement  and  attitude,  and  technical.  Given 
the  structure  of  the  inquiry,  one  anticipated  outcome  was  that  the  resulting  data  would  have  an  emphasis  on 
learner  related  issues,  a  result  which  was  supported.  Similar  to  studies  by  others  in  the  field  of  distance 
education  research  (Holmberg  &  Bakshi,  1992;  Walker  &  Hackman,  1992;  Kember,  Lai,  Murphy,  Siaw,  & 
Yu^n?  1992),  this  study  relied  on  a  survey  for  gathering  information  from  the  participants. 

At  the  University  of  Hawaii,  Small  Group  Instructional  Diagnosis  has  been  employed  for  the  past 
five  years  by  its  Center  for  Teaching  Excellence,  a  unit  within  the  Office  for  Faculty  Development  and 
Support.  At  the  mid  point  of  the  semester  in  traditional,  intact  classes  throughout  the  campus,  trained 
faculty  and  teaching  assistants  from  the  Center  assist  faculty  in  assessing  the  effectiveness  of  their  courses 
by  obtaining  student  perceptions  about  their  learning.  In  addition,  since  1969,  when  the  University  of 
Hawaii  offered  the  first  college  credit  course  delivered  by  satellite,  the 'university  has  been  developing  its 
interactive  television  facilities.  The  Hawaii  Interactive  Television  System  (HITS)  currently  delivers  a  wide 
range  of  courses  at  the  undergraduate  and  graduate  levels  using  a  technologically  advanced  delivery  system 
that  incorporates  two-way  audio  and  video  at  its  origination  site  on  the  island  of  Oahu  and  at  its  receive 
sites  on  the  neighbor  islands  of  Kauai,  Hawaii,  and  Maui.  To  date,  evaluation  of  HITS  courses  and 
programs  has  been  accomplished  through  written  responses  provided  by  both  students  and  HITS  instructors 
in  end-of-semester  questionnaires.  None  have  incorporated  the  unique  interactive  feature  offered  by  HITS 
into  the  evaluation  process. 

With  the  SGID  model  and  HITS  at  the  University  of  Hawaii,  the  resources  were  available  for 
undertaking  a  preliminary  investigation  of  the  effectiveness  of  applying  an  interactive  classroom  evaluation 
procedure,  SGID,  to  introductory  courses  in  a  cluster  of  five  Department  of  Educational  Technology 
(ETEC)  graduate  courses,  entitled  Telecommunications  and  Technology  for  Teachers  (T^),  delivered  via 
interactivf    ievision.  Although  the  SGID  model  has  been  used  in  traditional  class  settings,  it  had  not  been 
used  in  a  distance  education  class  at  the  University  of  Hawaii  and  no  instances  of  its  use  in  other  distance 
education  classes  were  discovered. 

The  department  faculty  determined  to  apply  the  SGID  at  mid-semester  to  the  T^  program  to  obtain 
timely  feedback  on  the  program  and  to  assess  the  effectiveness  of  this  evaluation  technique  in  a  distance 
education  setting. 

Questions  under  investigation  were: 

Question  1 :  Would  a  modification  of  the  SGID,  delivered  as  a  computer-mediated  communication, 
provide  data  of  sufficient  quality  and  quantity  for  use  in  a  study  of  SGID  as  a  distance  education  evaluation 
process? 

Question  2:  Would  a  cohort  of  graduate  students  enrolled  in  a  cluster  of  distance  education  courses 
differ  in  their  perceptions  of  a  course  evaluation  process,  the  Small  Group  Interactive  Diagnosis?  Would 
those  perceptions  be  influenced  by  receive  site  location;  by  effects  of  two-way  audio  and  video  during  group 
reports  to  the  entire  class;  by  the  effects  of  video  graphics  used  for  ranking  results  of  small  groups  reports; 
and  by  the  large  group  sizes  that  some  sites  may  have  during  the  SGID? 

Question  3:  Would  the  results  of  the  process  be  available  for  program  administrators  and 
instructors  in  a  timely  manner,  allowing  for  program  and  course  revisions? 
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Methods  and  Procedures 

Subjects 

The  population  lor  this  study  consisted  of  a  cohort  of  fifty  Department  of  Education  teachers  and 
specialists  enrolled  in  the  first  two  of  five  graduate  level,  courses  that  comprised  the  Telecommunications 
and  Technology  for  Teachers  (T3)  Program.  The     courses  were  scheduled  two  nights  per  week  for  the  fall 
and  spring  semesters  of  1993-1994,  with  the  final  course  scheduled  in  the  first  1994  summer  session. 
While  these  courses  were  part  of  the  graduate  program  offered  within  the  Department  of  Educational 
Technology  at  the  University  of  Hawaii,  they  did  not  of  themselves  comprise  an  official  program  of  study. 
The  Department  of  Education  acknowledged  the  completion  of  the  five  courses  with  a  T3  Certificate  of 
Completion,  which  led  to  the  informal  designation  of  the  courses  as  the  T3  Program. 

Students  were  located  at  three  remote  sites:  the  islands  of  Maui,  Kauai,  and  Hawaii,  students 
were  also  present  at  the  main  campus  transmission  site  on  the  island  of  Oahu.  Ninety  percent  of  the 
cohort,  45  students:  21  from  the  on-campus  site  and  24  from  the  remote  sites,  participated  in  the  Small 
Group  Instructional  Diagnosis  during  the  ninth  week  of  the  15  meeting  cycle.  Following  the  SGID 
protocol,  a  trained,  external  facilitator  met  with  the  students  for  an  entire  class  period  without  the  presence 
of  the  instructors.  The  students  actively  analyzed  the  effectiveness  of  the  first  two  courses  in  the  T3 
Program  by  responding  to  three  questions  about  what  helped  their  learning  in  the  program,  what  hindered 
their  learning  in  the  program,  and  what  changes  they  would  suggest  for  the  program. 

At  the  conclusion  of  the  two  courses,  the  participants  were  asked  to  evaluate  this  SGID  process 
using  a  version  of  SGID  modified  for  delivery  as  a  computer-mediated  communication. 

Eighty-seven  percent  of  the  45  mid-semester  participants  participated  in  this  evaluation:  19  from 
the  on-campus  site  and  20  from  the  remote  campuses.  These  39  students  formed  the  sample  for  this  study. 
Characteristics  that  the  sample  have  in  common  with  all  distant  education  populations  are  their  adult  status 
and  the  predominance  of  females,  82%.  Their  wide  ranging  ethnic  backgrounds,  while  typical  for  the  state 
of  Hawaii,  are  not  typical  of  other  distance  education  students.  This  study  is  limited  by  its  intact 
population. 

Measures 

Computer  Mediated  Communication  SGID:  The  quantity  and  quality  of  responses  to  the  on-line 
delivered  survey  questions  were  assessed  by  amount  and  quality  of  information  conveyed.  A  response  rate 
indicative  of  75%  was  established  as  a  way  to  assess  the  quantity  of  the  returns  (Seldin,  1988).  Response 
quality  was  assessed  by  the  degree  to  which  the  respondents  provided  usable  information  for  each  of  the 
three  questions. 

Perceptions  of  the  SGID:  To  assess  the  extent  to  which  the  subjects  differed  in  their  perceptions 
of  SGID,  a  modified  version  of  the  SGID  process  itself  was  used.  As  a  qualitative  measure,  it  has  no 
reliability  coefficients;  its  construct  validity  is  attested  to  by  use  by  others  since  1979  (Clark  &  Bekey, 
1979;  Seldin,  1988;  Theall  &  Franklin,  1990).  The  overall  impact  of  the  SGID  was  assessed  through  a 
comparison  of  the  number  of  responses  for  each  of  the  three  questions  in  relation  to  positive  responses. 

The  three  questions  designed  to  elicit  open-ended  responses  about  the  positive  and  negative  aspects 
of  the  evaluation  process,  as  well  as  to  elicit  suggestions  for  changes  in  SGID,  were  sent  to  the  on-line 

T3 

discussion  list.  The  request  and  survey  were  transmitted  to  the  T3  students  on  the  discussion  list  that  had 
been  established  and  extensively  used  throughout  both  courses.  To  participate  in  the  study,  the  students  had 
to  have  participated  in  the  SGID  administered  at  mid-  semester.  Responses  were  sent  to  the  primary 
researcher.  Attesting  to  their  interest  in  evaluating  the  process,  in  addition  to  responding  to  the  researcher, 
over  40%  of  the  respondents  elected  to  send  a  copy  of  their  responses  to  the  T3  discussion  list,  following 
the  T3  program  protocol  of  sharing  information  of  general  interest  with  all  participants. 

Patterns  of  concepts  and  frequency  of  words  and  phrases  related  to  the  variables  of  site,  effect  of 
delivery  mode/graphic  display,  and  group  size  were  set  as  the  way  to  interpret  responses  related  to  those 
variables. 

Program  Impact:  To  study  the  impact  of  the  results  of  the  SGID  on  the  T3  Program,  the  log  of 
administrative  actions  occurring  after  reception  of  the  SGID  results  was  analyzed  by  types  of  actions. 
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Question  I :  To  assess  the  viability  of  using  the  on-line  generated  responses,  the  response  rate  was 
computed  for  both  inter  group  and  total  group  and  compared  to  the  minimal  desired  rate  of  75%  (Seldin, 
1988). 

Question  2:  To  assess  differences  in  the  perceptions  of  the  distance  education  students  at  both  the 
on-campus  and  remote  sites,  student  responses  relevant  to  the  three  SGID  questions  were  carefully  reviewed, 
separated  into  discrete  phrases,  coded,  and  entered  into  a  database  and  analyzed. 

During  the  preliminary  analysis  of  the  phrases,  a  conceptual  framework  emerged  that  appeared 
related  to  the  three  interactions  identified  by  Moore  (1989)  regarding  distance  education  settings.  His 
"learner-content  interaction"  emerged  as  "learner-format  interaction"  for  this  analysis.  Participants' 
responses,  such  as,  "The  three  questions  were  to  the  point,"  were  placed  in  this  category.  Moore's  "learner- 
instructor  interaction"  emerged  as  "learner-facilitator  interaction"  with  the  facilitator  referring  to  the  person 
who  conducted  the  SGID.  Statements,  such  as,  "having  a  patient  and  objective  facilitator,"  were  identified 
as  learner-facilitator  interactions.  No  changes  in  the  semantics  for  Moore's  "learner-learner  interaction" 
category  were  deemed  necessary.  Statements,  such  as,  "being  able  to  see  the  number  of  students  that  agreed 
with  my  feelings,"  were  identified  as  learner-learner  interactions. 

The  plausibility  of  this  coding  was  tested  with  an  expert  panel,  consisting  of  the  researchers  and  an 
external  evaluator.  The  researchers  provided  deep  understandings  from  their  immersion  in  the  setting;  the 
independent  expert  provided  an  unbiased  view  and  knowledge  as  an  evaluation  specialist.  Response 
categorizations,  made  independently  by  the  expert  panelists  for  all  items,  revealed  a  high  degree  of  inter-rater 
agreement  with  the  fit  of  Moore's  modified  typology.  Those  items  not  agreed  upon  were  discussed, 
resulting  in  mutual  agreement  on  the  categorization  of  all  items. 

Further  analysis  within  the  typology  was  carried  out  with  like  words  and  phrases  within  the 
database.  For  example,  "continue"  and  "don't  change"  were  considered  synonymous. 

Question  3:  To  analyze  program  activity  generated  by  the  results  of  the  SGID,  related  occurrences 
were  analyzed  by  investigating  the  data  arrayed  on  a  timeline. 

Results 

Question  1 :  The  results  revealed  a  response  rate  of  87%  for  all  sites,  with  an  on-campus  site 
response  rate  of  90%  and  a  remote  site  response  rate  of  83%.  All  response  rates  exceeded  the  standard  set  for 
a  minimal  75%  return  rate,  thus,  providing  sufficient  quantity  of  data  for  analysis.  As  indicated  in  Table  1 , 
the  data  was  relevant  to  the  three  SGID  questions.  Of  the  213  types  of  responses,  only  6  items  were  not 
applicable  to  the  three  qurstions:  "What  helped  the  evaluation  process?"  "What  made  the  evaluation  process 
difficult?"  "What  recommendations  do  you  have  for  changes  for  the  mid-semester  Small  Group  Instructional 
Diagnostic  process?"  Both  the  overall  high  return  rate,  87%,  and  the  number  of  usable  items,  207  out  of 
213,  indicated  that  the  computer  mediated  communication  was  an  effective  process  forgathering  data  for  this 
study  from  participants  who  were  all  active  in  their  use  of  on-line  communications.  These  participants 
were  characterized  as  having  a  high  level  of  comfort  in  sending  messages  and  to  each  other,  to  their 
instructors,  and  to  the  T^  discussion  list.  The  response  rate  may  hot  be  replicable  for  new  on-line  users  or 
for  users  *ho  are  not  used  to  corresponding  with  their  instructor  in  this  manner. 

Question  2:  To  determine  any  differences  between  the  on-campus  and  remote  site  participants, 
within  group  percentages  for  responses  were  calculated  and  compared.  As  indicated  in  Figure  1,  few 
differences  were  reflected  by  respondents.  Approximately  50%  of  the  comments  referred  to  the  helpfulness 
of  the  evaluation  process,  regardless  of  the  location  of  the  respondent.  Less  than  26%  referred  to  hindrances, 
or  need  for  modifications,  for  SGID.  An  examination  of  Table  1  reveals  that  some  differences  do  appear 
within  sub  categories  of  the  three  main  categories,  with  fhe  most  marked  differences  appearing  in  relation  to 
hindering  interactions.  At  the  remote  sites,  74%  of  the  responses  given  by  the  participants,  such  as, 
"Towards  the  end  of  the  session,  we  felt  pressed  for  time,"  were  categorized  as  being  learner-process 
interactions,  in  comparison  to  only  50%  of  this  type  of  comment  being  given  by  participants  at  the  on- 
campus  site. 

A  new  type  of  sub  category  emerged;  one  reflecting,  "no  problems".  While  actual  numbers  are 
small  for  meaningful  interpretations  of  this  subcategory,  it  is  interesting  to  see  fhe  patterns  that  appeared 
within  both  the  hindering  interactions  and  changing  interactions  categories.  At  the  on-campus  site,  a  total 
of  eight  comments,  such  as,  "I  don't  think  that  there  was  anything  that  made  the  process  difficult,"  in  the 
hindrances  group  and  eight,  such  as,  "I  would  recommend  we  continue  to  have  mid-semester  diagnostic 


feedback  sessions,"  in  ihe  modifications  group  were  made.  In  both  instances,  however,  only  two  or  less 
were  made  at  a  remote  site.    The  remote  site  also  offered  1 8%  more  comments  on  ideas  for  changes  related 
to  learner-process  interactions  than  did  the  on-campus  group.  These  comments  included,  "Perhaps  a  faster 
way  of  counting  the  'votes'  for  each  of  the  items.  Maybe  using  the  fax  machine  to  facilitate  this."  "Also, 
perhaps  limiting  each  group  to  3-5  answers  to  each  question.  I  think  this  would  eliminate  some  answers 
that  were  repetitive  and  some  that  were  'off  the  wall'  and  subsequently  received  few  votes." 

After  categorization  of  items  by  the  panel  into  types  of  interactions,  like  items  were  further 
collapsed  to  create  items  of  similar  concepts,  so  that  "professional  manner",  "professionally  done", 
"excellent  job",  and  ,  "leadership  skills"  were  categorized  as  overall  skills  of  the  facilitator.  These  items 
were  ranked  by  sites  based  on  frequency  counts  within  groups.  The  results  of  this  ranking,  displayed  in 
Tables  2  through  4,  show,  that  regarding  helpful  interactions,  strong  agreement  was  reflected  at  all  sites 
regarding  the  excellent  overall  skills  of  the  SGID  facilitator.  This  item  was  mentioned  more  than  any  other 
for  all  groups,  as  well  as  by  the  remote  site  group.  For  the  on-campus  group,  this  item  was  in  a  three-way 
tie  for  first  place  with  "the  facilitator's  ability  to  create  a  supportive  atmosphere"  and  in  the  learner- learner 
interaction  subcategory,  with  "professionalism  of  the  group".  Tables  2  through  4  provide  a  comparison  of 
items  that  respondents  cited  most  frequently.  Unanimity  was  reached  across  groups  on  hindering 
interactions  within  the  learner-process  interaction  subcategory  with  "the  amount  of  time  for  the  process" 
being  the  most  frequently  mentioned  item  being  perceived  as  a  problem  and,  similarly,  in  the  change 
interactions  category  with  "conducting  earlier  in  the  semester". 

An  analysis  of  the  number  of  times  that  specific  references  to  any  constraints  imposed  on  the 
evaluation  process  by  the  effects  of  two-way  audio  and  video  on  the  group  reports  and  of  the  video  graphics 
used  to  display  the  group  reports  for  voting,  showed  that  these  items  received  specific  mention  only  once 
each  at  the  remote  sites  and  no  mention  at  the  on-campus  site.  The  remote  site  comments  were:  "It  was 
difficult  not  being  in  the  same  room  because  comments  were  made  that  [were]  not  stated  openly."  "TV 
made  it  difficult  to  keep  track  of  all  the  comments  since  all  of  the  writing  had  to  occur  on  the  elmo  [visual 
presenter].  Still  this  is  a  limitation  that  cannot  be  helped."  One  positive  comment  was  received  from  a 
remote  site  in  regard  to  the  visualization  of  the  group's  ideas:  "I  also  think  it  helped  that  [the  facilitator] 
wrote  down  all  the  ideas  and  paraphrased  to  ensure  agreement." 

There  were  more  items  related  to  the  large  group  sizes  with  five  comments,  6.8%,  from  the  on- 
campus  site  regarding  learner-learner  interaction  problems:  "I  did  notice  some  people  did  not  volunteer 
much  for  discussion.  It  was  their  choice  though."  The  remote  sites  offered  three  negative  comments,  5.3%, 
in  this  regard.  A  typical  comment  was,  "The  group  was  a  little  too  large."   At  the  same  time,  both  the  on- 
campus  and  remote  site  participants  made  ten  comments,  13.5%  and  17.5%  respectively,  regarding  the 
helpfulness  of  the  learner-learner  interactions:  "I  liked  how  it  was  directed  and  that  we  were  broken  up  into 
groups  so  our  'voices'  could  be  heard  collectively".  "Group  memory  kept  the  group  focused  on  the  tasks." 
These  results  indicate,  that  for  these  participants,  the  issue  of  technical  issues  and  group  size  were  not 
predominant  concerns. 

Question  3:  To  analyze  the  data  in  relation  to  question  3,  to  see  if  the  results  from  the  Small 
Group  Instructional  Diagnosis  provided  results  in  a  timely  manner  to  allow  for  program  and  course 
revisions,  the  activities  that  the  process  engendered  were  plotted  on  a  timeline.  Figure  2  shows  the  increase 
in  activity  immediately  prior  to  the  administration  of  the  SGID,  and  the  subsequent  activity  following  the 
process.  Of  particular  note  is  the  appearance  of  the  Task  Force  on  the  Technology  Specialist.  This  Task 
Force,  charged  with  the  examination  of  the  actual  role  of  the  proposed  new  Department  of  Education  (DOE) 
specialization,  had  been  discussed  at  prior  meetings,  but  the  SGID  provided  the  impetus  to  bring 
stakeholders  from  both  the  ETEC  Department  and  the  DOE  together  to  identify  the  skills  needed  by 
someone  in  this  proposed  position.  The  ETEC  Department  also  used  the  results  of  the  SGID  for  long- 
range  and  short-range  planning.  Based  on  SGID  data  on  course  scheduling,  courses  for  the  following 
semester  were  scheduled  in  tandem  rather  than  concurrently.  For  the  long  term,  the  ETEC  Department  began 
re-examining  the  courses  selected  for  the  program  in  light  of  the  new  scheduling,  with  plans  for  a  seminar 
session  to  be  interwoven  throughout  the  courses. 


Discussion 

In  this  study,  a  model  for  evaluating  two  distance  education  courses  that  formed  part  of  a  program 
of  study,  was  investigated.  The  range  of  data  obtained  from  the  distance  education  participants  supported  the 
recommendation  from  a  study  of  seven  distance  education  courses  in  Hong  Kong  (Kember  et  al.,  1992)  that 
others  include  more  than  one  course  in  their  evaluation  to  provide  a  broader  range  of  input.  Although  the 
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Kember  et  al.  study  used  a  mix  of  qualitative  and  quantitative  methods,  their  qualitative  data  was  used  more 
extensively  in  their  analysis. 

Time  constraints  posed  limits  on  the  manner  in  which  this  computer  mediated  communication 
version  of  the  SGID,  used  for  gathering  data  on  the  actual  SGID,  was  executed.  The  facilitator  role  of 
restating  and  clarifying  items  which  could  have  been  attempted  on  the  class  discussion  list,  was  omitted  in 
the  on-line  version  because  of  the  time  constraints  imposed  by  the  approaching  end  of  the  semester. 

This  study  provided  a  connection  to  the  everyday  world  of  educational  practice.  The  high  response 
rate  to  the  on-line  SGID,  in  less  than  three  weeks  by  the  Distance  Education  students,  reflected  the  appeal  of 
this  modified  version  for  data  gathering,  supporting  Reinharz's  endorsement  for  using  "special  techniques 
that  are  "unobtrusive  measures"  (1 993).  His  support  for  having  good  pre-conditions  for  a  specific  method 
was  supported  by  the  familiarity  and  positive  attitudes  of  the  participants,  toward  on-line  communication. 
Lauzon  ( 1992)  also  supports  on-line  communication  as  an  excellent  medium  for  meaningful  interactions 
between  learners  and  facilitators,  instructors,  and  each  other.  A  version  of  SGID  modified  for  use  as  a 
computer  mediated  communication  did  provide  useful  information  for  this  qualitative  study. 

Analysis  of  the  comments  was  carried  out  by  the  expert  panelists,  using  the  students'  language  to 
tell  what  was  considered  important,  requiring  reliance  "on  their  own  intelligence,  experience  and  judgments" 
(Patton,  1987,  p.  154).  The  use  of  Moore's  typology  (1989)  that  emerged  from  the  analysis  of  the 
authentic  voices  of  the  participants,  gave  structure  to  the  analysis  of  the  items  and,  thus,  guidance  for 
replicating  this  study. 

Identifying  a  modification  of  Moore's  typology,  instead  of  creating  their  own,  also  gave  the 
authors  a  way  of  portraying  the  students'  views  in  a  manner  that  addressed,  to  some  degree,  the  potential  for 
bias  that  could  arise  for  the  authors  in  their  dual  roles  as  participants  throughout  the  SGID  process  and  as 
authors  of  the  study.  The  external  member  of  the  codifying  panel  attested  to  the  validity  of  applying  the 
adapted  version  of  Moore's  typology. 

The  emergence  of  three  categories  of  the  interactions  of  learner-format  ("learner-content"  in 
Moore's  categorization),  learner-facilitator  ("learner-instructor"  according  to  Moore)  and  learner-learner 
supports  the  distinctions  being  made  in  the  field  of  distance  education  of  the  importance  of  identifying 
typologies  for  interactions.  Such  schemata  lead  to  improved  communication  and  study  within  the  field 
(Moore,  1989).  It  is  of  note  that  two  types  of  responses  did  not  fit  his  typology;  these  responses  are  tne 
items  that  were  cited  when  participants  were  asked  to  name  hindrances  or  changes  to  interactions.  The 
responses  of  "no  hindrances"  and  "no  changes"  were  noted  in  separate  categories  established  to  accommodate 
this  type  of  response. 

The  data  from  this  study  suggests  use  of  the  Small  Group  Instructional  Diagnosis  as  an  effective 
evaluation  model  for  distance  education  students  enrolled  as  a  cohort  in  graduate  level  education  courses. 
More  than  50%  of  the  response  items  related  to  factors  that  helped  the  process,  regardless  of  the  site;  with 
only  21%  of  the  items  at  the  remote  sites  and  16%  of  the  items  at  the  on-campus  site  related  to  items  that 
interfered  with  the  process.  This  finding  supports  an  earlier  evaluation  (Clark  &  Bekey,  1979)  of  the 
process.  In  Clark  &  Bekey's  evaluation  of  die  effectiveness  of  SGID  for  students  in  a  traditional  educational 
setting,  80%  of  the  participants  stated  that  SGID  was  useful  when  asked  to  respond  in  either  the  affirmative 
or  the  negative. 

While  a  little  less  than  7%  of  the  distance  education,  campus-based  students  registered  their 
enthusiasm  for  the  SGID  in  statements  on  "continuing/not  changing  the  process"  and  in  requests  "for  using 
the  process"  in  their  roles  as  teachers,  no  similar  remarks  were  noted  from  the  remote  site  students.  The 
remote  site  participants  made  quantifiably  more  hindering  interaction  statements  than  the  on-campus 
participants,  although  both  types  of  participants  identified  "the  amount  of  time  for  the  process",  "time 
limits",  and  "peer  pressure"  as  their  top  concerns.  The  remote  site  participants  expressed  a  greater  range  of" 
comments,  from  statements  such  as  "the  voting  didn't  quite  express  the  ideas",  "negative  feelings",  and 
"versions  varied  by  site".  This  wider  range  could  be  indicative  of  differences  that  may  have  occurred  from 
aggregating  the  responses  from  three  different  remote  sites  into  the  sole  category  of  "remote  site"  and  from 
two  of  the  remote  sites  having  less  options  for  forming  varied  groups  than  the  on-campus  site  and  the  third 
remote  site,  both  of  which  could  accommodate  three  groups  of  seven  . 

The  size  of  the  groups,  most  of  which  consisted  of  seven  participants,  and  as  such  violated  by  one 
person  the  maximum  number  recommended  for  groups  in  the  SGID  (Clark  &  Bekey,  1979),  may  account 
for  the  concerns  expressed  about  time.  The  additional  person  may  impact  the  amount  of  time  available  for 
optimal  learner  to  learner  interactions. 

Although  reports  of  SGID  may  omit  mention  of,  or  not  stress,  administration  by  a  trained 
facilitator  (Theall  &  Franklin,  1990;  Seldin,  1988),  the  presence  of  a  trained,  external  facilitator  was  the 
factor  students  in  this  study  most  strongly  supported;  a  factor  also  identified  as  critical  in  the  work  of  Clark 
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and  Bekey  (1979).  Having  the  opportunity  to  express  ideas;  anonymity;  satisfaction  in  knowing  the 
results  would  be  listened  to  by  the  stakeholders  in  the  program;  and  having  clear  guidance  for  the  process 
were  themes  that  appeared  in  comments  in  both  this  study  and  in  Clark  and  Bekey's  work.  In  regard  to  the 
closely  related  topic  of  interaction  between  learner  and  instructor,  Dillon,  Gunawardena,  &  Parker  (1992) 
found  that  such  interactions,  during  class  time,  were  viewed  as  the  most  important  form  of  interaction  with 
the  distance  education  students  they  studied. 

Although  Clark  and  Bekey's  procedure  for  administering  the  SGID  requires  that  the  facilitator's 
visit  not  be  announced  in  advance  to  avoid  student  absences  by  some  who  might  feel  strongly  about  the 
class,  unlike  Clark  and  Bekey's  process,  SGID  at  the  University  of  Hawaii,  provides  prior  notice.  This 
study  followed  that  pattern,  with  no  change  noted  in  the  attendance  patterns.  The  participants  in  this  study 
provided  further  support  for  the  idea  of  prior  notice,  by  recommending  that  the  three  questions  be 
distributed  prior  to  the  process. 

Students  offered  few  comments  on  the  audio  and  video  deliveiy  system,  despite  Walker  & 
Hackman's  finding  (1992)  that  clear  audio  and  video  were  significant  factors  for  distance  education  students. 
Given  the  ease  and  technical  sophistication  with  which  the  HITS  delivery  system  functions  at  the 
University  of  Hawaii,  this  finding  was  not  unexpected  for  these  students. 

Following  guidelines  for  using  evaluations  for  improving  instruction  (Maitino,  1992;  Theall  & 
Franklin,  1990),  the  direct  effects  of  the  SGID  results  were  the  inter  and  intra  university  and  school 
departmental  meetings  to  hear  the  results  of  the  SGID  and  to  plan  for  recommended  changes.  While  many 
factors  figured  in  the  advancement  of  tentative  plans  for  developing  a  technology  specialist  certificate  within 
the  Department  of  Education  and  the  concomitant  graduate  level  training,  an  examination  of  the  timeline  in 
Figure  2  shows  the  close  proximity  of  the  major  thrust  of  that  work  to  the  SGID.  The  university  and 
school  stakeholders  focused  formally  on  a  Task  Force  charged  with  identifying  needs  and  defining  the  role 
for  such  a  professional  after  receiving  the  results  of  the  evaluation  process. 


Implications 

While  some  studies  show  linkages  of  student  achievement  to  students'  ratings  of  courses  (Cohen, 
198 1),  others  (Byrne,  1992)  question  the  validity  of  using  such  data,  because  of  the  dangers  inherent  in 
obtaining  reliable  estimates  of  coefficients  from  small  samples,  particularly  if  employment  decisions  for 
instructors  are  based  on  such  results.  Given  the  small  number  of  participants  and  lack  of  legal  access  to 
student  grades,  this  study  made  no  attempt  to  examine  the  relationship  between  the  students'  ratings  and 
their  academic  achievement  in  the  courses.  Future  researchers  with  larger  samples  may  wish  examine  such 
quantitative  data  in  relation  to  the  qualitative  data  the  SGID  provides.  The  results  of  this  study  should  be 
interpreted  with  caution  as  a  descriptive  study  of  informal  linkages. 

Others  may  wish  to  draw  upon  the  ease  with  which  the  methods  employed  in  this  study  obtained 
cooperation  from  the  participants  and  with  the  ability  to  gather  a  wide  range  of  perspectives. 

The  pattern  that  emerged  from  this  research  is  the  critical  role  the  facilitator  plays  in  SGID. 
Future  studies  need  to  further  investigate  the  significance  of  this  effect. 

In  light  of  the  recognized  importance  of  perceived  interactivity  to  distance  education  student 
satisfaction  (Fulford  &  Zhang,  1994),  future  studies  may  also  wish  to  investigate  the  relation  of  SGID  in 
contributing  to  student  perceptions  of  interactivity  within  their  distance  education  environment. 

The  SGID  provides  a  structure  that  meets  guidelines  for  obtaining  student  ratings  for 
improvement.  Seldin  (1988)  has  defined  these  guidelines  as:  asking  diagnostic  questions;  having  open- 
ended  response  format  support  feedback  on  an  infinite  variety  of  factors;  mainti  ining  confidentiality  for 
instructors;  providing  questions  that  students  are  capable  of  answering;  encouraging  instructors  to  actively 
participate  by  adding  any  specific  questions  they  may  have;  being  able  to  use  other  evaluation  instruments; 
supporting  comprehensive  student  involvement;  and  providing  written  comments.  SGID  accommodates  all 
these  guidelines. 

The  assessment  of  student  attitudes  toward  their  distance  education  milieu  is  one  of  the  factors  that 
continues  to  appear  in  the  literature  on  distance  education  (Thorpe,  1988;  Wagner,  1993).  This  study 
attempted  to  link  research  and  evaluation  of  distance  education  in  the  area  of  student  learning  in  relation  to 
graduate  level  in-service  teachers  located  on  four  islands  in  Hawaii. 

Given  the  factor  that  college  courses  delivered  by  television  reach  more  adult  students  at  more  colleges  than 
any  other  type  of  technology-based  course  (Brock  1991 ),  there  is  a  need  to  continue  investigating  evaluation 
methods  for  such  distance  education  courses.  To  fill  the  continuing  need  for  rigorous  practical  research  on 
assessment  for  distance  education  (Holmberg,  1991),  future  studies  with  diverse  distance  education 
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populations  could  be  designed  to  employ  Small  Group  Instructional  Diagnosis,  which,  with  its  potential 
for  gaining  broad  insights  into  students  perceptions  and  attitudes  abrjt  distance  education  courses,  in 
combination  with  quantitative  measures,  has  the  potential  for  providing  additional  insights  into  student 
satisfaction  in  distance  education  settings. 
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Table  i 

Typology  of  Student  Reactions  to  Small  Group  Instructional  Diagnosis  Model  by  Sites 

 Sites  


Types  of  Interaction  in  Evaluation 

Process  All  Campus  Remote 


Helpful  Interactions 

Learner-Process 

44 

(41.0) 

23 

(44.2) 

21 

(37.5) 

Learner-Facilitator 

44 

(41.0) 

19 

(36.5) 

25 

(44.6) 

Learner-Learner 

(18.5) 

(iy.z) 

1 U 

(17.2) 

Hindering  Interactions 

Learner-Process 

28 

(62.2) 

ii 

(50.0) 

17 

(74.0) 

Learner-Facilitator 

1 

(2.2) 

0 

(0.0) 

1 

(4.3) 

Learner-Learner 

8 

(17.7) 

5 

(22.7) 

3 

(13.0) 

None 

8 

(17.7) 

6 

(27.3) 

2 

(8.7) 

Change  Interactions 

Learner-Process 

46 

(85.2) 

24 

(77.4) 

22 

(95.7) 

Learner-Facilitator 

0 

(0.0) 

0 

(0.0) 

0 

(0.0) 

Learner-Learner 

0 

(0.0) 

0 

(0.0) 

0 

(0.0) 

No  Changes 

8 

(14.8) 

7 

(22.6) 

1 

(4.3) 

Note,  n  is  based  on  number  of  items,  not  respondents,  n  =  207  for  all  items. 

n  =  105  for  on-campus  items,  n  =  102  for  remote  campus  items.  Percentages  figured  on  100%  for  the 
three  main  types  of  interaction  appear  in  parentheses. 
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Table  2 

Ranking  of  Student  Comments  Concerning  Distance  Education  Evaluation  Model  Related  to  Types  of 
Interaction  bv  Site:  Helpful  Interactions 

 Rank  by  Sites  

Types  of  Interaction  in  Evaluation  Process  All         Campus  Remote 


Helpful  Interactions 

Learner-Process 

Structured  questions 

Opportunity  to  express  ideas 

A  third  party  guide 

Non-threatening  environment 

Meaningful  dialogue  focused  on  improvement 

Learner-Facilitator 

Overall  Skills:  professional/excellent/leadership 

Atmosphere  Creation  Ability:  supportive  of 
participation/neutral/comfortable/opportunities  to  respond 
Facilitation  Skills:  explicit  in  instructions/everyone  kept  on  track 
Mediation  Skills:  objective;  effective  mediator/  honest/non- 
judgmental 

Leamer-Leamer 

Hearing  ideas  of  others/group  memory/collective 
voices/unanimity/learning  from  each  other 
Professionalism  of  groups 


aMost  frequently  mentioned  item  within  all  types  of  Helpful  Interaction  categories.  Rank  based  on 
frequency  count  of  items  for  All  Sites,  n  =  207;  for  On-Campus  Sites,  n  =  105;  for  Remote  Sites,  n  = 
102. 


2 
2 
2 


2 


la 


1 
2 


2 
la 


1 

2 


Table  3 

Ranking  of  Student  Comments  Concerning  Distance  Education  Evaluation  Model  Related  to  Types  of 
Interaction  by  Site:  Hindering  Interactions 

 Sites  

Types  of  Interaction  in  Evaluation  Process  All  Campus  Remote 

Hindering  Interactions 
Learner-Process 


Amount  of  time  for  process 

la 

la  la 

Time  Limits 

2 

1  1 

Personal  feelings 

2 

2 

Learner-Facilitator 

1 

Facilitator  monitor  time 

1 

Leamer-Leamer 

1  1 

Peer  pressure 

1 

Feelings  about  self 

2 

2 

No  Problems 

1  1 

Not  difficult 

1 

aMost  frequently  mentioned  item  across  all  types  of  Hindering  Interactions 
categories.  Rank  based  on  frequency  count  of  items  for  All  Sites,  n  =  207;  for  On- 
Campus  Sites,  n  =  105;  for  Remote  Sites,  n  =  102. 
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Table  4 


Ranking  of  Student  Comments  Concerning  Distance  Education  Evaluation  Model  Related  to  Types  of 
Interaction:  Change  Interactions 


Types  of  Interaction  in  Evaluation  Process 


All 


Sites 


Campus 


Remote 


Changes  Interactions 
Learner-Process 

Conduct  earlier  in  the  semester 

Provide  the  questions  prior  to  the  session 
No  Changes 

Continue/don't  change 
Would  like  to  use  format 


la 

2 

1 

2 


la 
2 


la 
2 


aMost  frequently  mentioned  item  across  all  types  of  Change  Interactions  categories.  Rank  based  on 
frequency  count  of  items  for  All  Sites,  n  =  207;  for  On-Campus  Sites,  n  =  105;  for  Remote  Sites  n  = 
102. 


Figure  1.  Percentage  Distribution  by  Sites  for  Items  on  Effectiveness  of  Small  Group  Instructional 
Diagnosis 
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Figure  2.  Timeline  of  Activities  in  Relation  to  Small  Group  Instructional  Diagnosis 
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Abstract 


This  study  examines  learner  perceptions  of  the  ease  of  asking  and  answering  quest;ons  with  the 
instructor  and  other  learners  in  the  interactive  television  classroom.  Using  a  Likert  scale  based  survey, 
learner  perceptions  were  collected  from  235  individuals  from  both  studio  and  remote  sites  in  eleven  courses 
offered  through  the  Hawaii  Interactive  Television  Classroom  (HITS).  Learners  from  the  remote  site 
locations  rated  the  interactive  TV  classroom  as  significantly  easier  for  asking  and  answering  questions  than 
the  traditional  setting,  while  learners  from  the  studio  groups  did  not  rate  the  two  situations  as  significantly 
different.  The  learner  location  (studio  or  remote)  or  type  of  communication  (ask  or  answer)  did  not  have  a 
significant  effect  on  learner  perceptions  of  the  ease  of  communication  with  the  instructor.  When  asking  and 
answering  questions  with  other  learners,  learners  at  the  remote  site  perceived  these  communication 
behaviors  as  significantly  easier  than  learners  at  the  studio  sites.  Overall,  learners  perceived  sending 
answers  to  be  easier  than  asking  questions  when  communication  with  learners  at  a  different  site,  but  they 
found  asking  questions  to  be  easier  than  answering  questions  when  communicating  with  learners  at  the 
same  site.  The  results  of  this  study  indicate  that  characteristics  of  the  interactive  television  classroom  can 
affect  learner's  perceptions  of  the  communication  process  and  instructors  need  to  pay  attention  to  the  effects 
of  these  characteristics  when  planning  TV  courses. 


Introduction 

Is  television  easier?  According  to  Salomon  (1983)  "learning  in  its  generic  sense,  greatly  depends 
on  the  differential  way  in  which  sources  of  information  are  perceived,  for  these  perceptions  influence  the 
mental  effort  expended  in  learning"  (p.42).  The  Salomon  model  proposes  that  the  amount  of  learning  via  a 
given  medium  is  proportional  to  the  amount  of  invested  mental  effort  (AIME).  Although  the  argument  is 
well  made  in  a  study  comparing  reading  and  educational  television,  Miller,  McKenna,  and  Ramsey  (1993) 
caution  against  extrapolating  these  findings  to  distance  education.  They  argue  that  two-way  interactive 
television  is  more  comparable  to  the  traditional  classroom  than  to  educational  television  and  should  have 
similar  results. 

Previous  research  comparing  the  traditional  and  the  interactive  distance  forms  of  instructional 
delivery  has  shown  no  significant  differences  in  student  learning  outcomes.  In  a  study  on  audio  tele- 
conferencing, Weingand  (1984)  found  students  in  both  traditional  and  distance  classes  had  similar 
achievement  levels.  Students  in  a  television  course  with  two-way  audio  also  showed  no  differences  in 
content  learning  (Haynes  &  Dillon,  1992).  Ritchie  and  Newby  (1989)  conducted  a  study  comparing  a  live 
interactive  television  course  and  a  classroom  lecture  course  and  found  student  achievement  to  be  the  same  in 
both  formats.  A  literature  review  by  Whittington  (1986)  supports  these  findings;  in  terms  of  student 
achievement  and  learning  outcomes,  the  traditional  and  interactive  distance  forms  of  instructional  delivery  do 
not  differ  significantly. 

Achievement  may  be  the  same,  but  research  comparing  interaction  in  the  traditional  and  distance 
education  settings  has  shown  students  interact  less  in  the  distance  classroom.  Students  at  distance  sites 
tended  to  ask  fewer  questions  (Barron,  1987)  and  interacted  less  with  the  instructor  (Haynes  &  Dillon, 
1992).  Ritchie  &  Newby  (1989)  found  students  in  a  traditional  setting  interacted  twice  as  often  and  used  a 
greater  variety  of  interaction  types  than  the  students  in  the  television  setting. 

Although  learning  outcome  is  an  important  measure  of  the  success  of  any  educational  program,  it 
is  not  necessarily  the  only  measure  of  success.  Hackman  and  Walker  (1990)  point  out  that  grades  alone  are 
not  a  complete  estimate  of  students'  learning  and  satisfaction.  Garrison  (1990)  suggests  that  the  quality  of 
an  educational  learning  experience  is  dependent  upon  two-way  communication.  The  teacher  should  not  just 
report  information,  and  the  student  should  not  be  a  passive  learner.  Quality  education  requires  opportunities 
through  dialogue  and  discussion  for  the  student  to  challenge  the  ideas  of  the  teacher  and  the  meaning  of  the 
information.  The  cognitive  speed  theory  (Fulford,  1993)  suggests  in  one-way  communication  only  half  of 
the  learner's  cognitive  capacity  is  being  utilized.  This  means  the  potential  for  distraction  is  increased. 
When  a  learner  is  not  engaged  in  a  situation,  renegade  thought  patterns  are  likely  to  occur.  Interaction  and 
anticipated  interaction  are  ways  of  keeping  learners  engaged. 
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Past  research  has  found  that  student  satisfaction  and  perceived  learning  are  affected  by  the 
availability  of  interaction.  When  students  had  the  opportunity  to  comment  on  lectures,  satisfaction  and 
perceived  learning  were  greater  (Hackman  &  Walker,  1990).  When  students  interacted  regularly  with  the 
instructor  and  other  students,  they  reported  increased  motivation  and  higher  quality  learning  experience 
(Garrison,  1990).  Fulford  and  Zhang  (1993)  found  that  students'  perceptions  of  high  levels  of  classroom 
interaction  corresponded  to  higher  levels  of  satisfaction. 

Interactive  forms  of  distance  education  such  as  audio  tele-conferencing  and  television  classrooms 
try  to  approximate  the  traditional  face-to-face  classroom.  These  systems  of  instructional  delivery  depend  on 
communication  technology  to  allow  interaction  (Garrison,  1990).  Garrison  contends  the  quality  of  distance 
education  programs  depends  on  the  technological  capability  of  the  system  to  provide  frequent  and  regular 
interaction.  Even  though  learning  outcomes  are  comparable,  distance  education  programs  may  be  inferior  to 
the  traditional  classroom  setting  when  interaction  is  inhibited. 

Kozma  (1991)  argues  that  a  learner  uses  the  medium  of  instruction  as  well  as  the  content  to  create 
knowledge  and  the  capabilities  of  the  medium  influence  the  learner's  information  processing.  This  means 
different  media  with  different  attributes  and  capabilities  can  produce  different  learning.  The  medium  being 
considered  in  this  study  is  the  two-way  interactive  television  form  of  instructional  delivery.  Kozma  (1991) 
suggested  the  learning  process  is  sensitive  to  external  environmental  characteristics  such  as  the  availability 
and  structure  of  information. 

Hackman  and  Walker  (1990)  suggest  even  though  learning  outcomes  are  the  same,  there  are  some 
fundamental  differences  between  the  two  forms  of  instructional  delivery.  Some  of  these  differences  are  overt 
and  may  relate  to  limitations  of  interaction.  In  the  television  course  with  two-way  interactive  video, 
students  at  the  remote  sites  and  in  the  studio  must  use  microphones  to  communicate  with  the  instructor  or 
with  students  at  other  sites.  Students  must  also  speak  to  the  camera  in  order  to  approximate  eye  contact  and 
must  see  themselves  on  the  television  screen  when  speaking.  Ritchie  (1993)  pointed  out  that  the  television 
does  not  show  the  instructor  at  all  times  and  all  of  the  students  are  not  shown  at  all  times.  In  the 
traditional  class,  students  always  see  the  instructor,  and  the  instructor  always  sees  the  students.  Under  the 
television  conditions  important  non-verbal  communication  such  as  body  language  can  be  limited  or  totally 
missing  (Fulford  &  Zhang,  1993;  Ritchie,  1993).  Some  students  report  uneasiness  in  front  of  the  camera, 
disliking  the  microphones,  and  high  anxiety  while  on  television  (Nahl,  1993).  These  barriers  may  create 
additional  mental  effort  and  possibly  make  interactive  television  harder  rather  than  easier. 

The  unique  features  of  the  interactive  television  classroom  are  components  of  the  communication 
process.  If  students  perceive  these  components  as  barriers  to  communication,  interaction  may  be  affected. 
Fulford  &  Zhang  (1993)  found  that  student  perceptions  of  overall  classroom  interaction  was  acritical 
predictor  of  satisfaction.  Notably,  they  found  that  students'  perceptions  of  personal  interaction  level  were 
not  significantly  related  to  actual  interaction  time  (Zhang  &  Fulford,  1994).  Hackman  and  Walker  (1990) 
found  students'  perceptions  of  learning  and  satisfaction  can  be  maximized  by  manipulating  design  features. 
These  studies  imply  student  perceptions  can  strongly  affect  interaction  and  satisfaction.  Salomon's  model 
of  mental  effort  may  be  more  relevant  to  student  perceptions  of  interaction  rather  than  achievement  in 
interactive  television. 

Moore  (1989)  describes  three  types  of  interaction:  learner-instructor,  learner-learner,  and  learner- 
content.  This  study  will  focus  on  the  first  two  types:  learner-instructor  and  learner-learner.  Because  of  the 
unique  situation  of  the  television  classroom,  both  interaction  types  can  be  divided  into  interaction  at  the 
same  site  and  interaction  between  different  sites.  In  the  traditional  classroom,  all  the  students  are  in  the 
same  room  with  the  instructor,  but  in  the  television  system  only  the  studio  site  students  are  in  the  same 
room  as  the  instructor. 

Instead  of  looking  at  overall  levels  of  interaction,  this  study  focuses  on  specific  behaviors:  asking 
questions  and  answering  questions.  These  behaviors  can  be  seen  as  important  starting  points  or  integral 
components  of  interactive  dialogue.  If  students  perceive  barriers  to  asking  and  answering  questions, 
interaction  may  be  seriously  curtailed. 

The  purpose  of  this  study  is  to  investigate  learner  perceptions  of  the  ease  of  asking  and  answering 
questions  in  the  TV  classroom  and,  how  these  perceptions  are  affected  by  the  location  of  the  individual 
sending  the  message,  the  location  of  the  individual  receiving  the  message,  and  the  type  of  message  being 
sent. 


Three  research  questions  have  been  posed: 

RQ1 .  Do  learners  at  the  studio  site  and  remote  sites  perceive  it  easier  to  ask  and  answer  questions 
in  the  traditional  class  or  the  interactive  TV  class? 

RQ2.  When  interacting  with  the  instructor,  does  the  learner's  location  (studio  or  remote)  affect  the 
perceived  ease  of  asking  and  answering  questions? 

RQ3.  When  interacting  with  other  learners  does  the  sender's  location  (studio  or  remote)  and  the 
receiver's  location,  the  destination  (same  or  different),  affect  the  perceived  ease  of  asking  and 
answering  questions? 


Methods 

Participants  were  235  students  enrolled  in  courses  taught  over  the  Hawaii  Interactive  Television 
System  (HITS)  HITS  is  an  electronic  communication  system  used  by  the  University  of  Hawaii  to  deliver 
courses  on  outer  islands  as  well  as  off-campus  sites  on  Oahu.  It  is  a  four-channel  interactive  closed  circuit 
television  network  that  uses  both  Instructional  Television  Fixed  Services  (ITFS)  and  point-to-point 
microwave  signals.  Participants  were  from  campuses  on  Manoa  (N=105),  Maui  (N=87),  Kapiolani 
Community  College  (N=14),  Kauai  (N=20),  and  Hilo  (N=9).  Students  at  the  Manoa  site  were  in  the  studio 
group  (N=105)  The  students  at  Maui,  Kapiolani  Community  College,  Kauai,  and  Hilo  were  in  the  remote 
group  (N=l  30)  Participants  were  enrolled  in  one  of  eleven  different  courses  offered  by  the  departments  of 
Educational  Administration,  Political  Science,  Educational  Technology,  Nursing,  Business  Administration, 
Public  Health,  Educational  Curriculum  and  Instruction,  and  Hawaiian  Studies.  The  data  were  collected 
during  the  Summer  Session  and  Fall  Semester  of  1994. 

A  questionnaire  was  developed  in  order  to  measure  learner  perceptions  of  the  case  of  sending 
messages  in  the  TV  classroom.  The  survey  had  three  sections:  interaction  with  the  instructor,  interaction 
with  learners  at  a  different  site,  and  interaction  with  learners  at  the  same  site.  The  heading  for  each  section 
was  "Compared  to  the  traditional  face-to-face  classroom,  I  feel  in  this  TV  claso  .."  The  items  were 
completions  of  this  statement.  Each  section  contained  three  parallel  items  about  the  ease  of  asking 
questions  and  three  parallel  items  about  the  ease  of  answering  questions  for  communication  with  the  subject 
of  that  section.  Students  responded  to  these  items  using  a  7-point  Likert  scale  with  1=  strongly  agree,4= 
neutral  and  7=  strongly  disagree.  For  analysis,  the  items  were  scored  so  lower  numbers  indicated  the  TV 
classroom  was  perceived  as  easier.  Each  subject  had  mean  sub-scores  for:  (a)  perceived  ease  of  asking 
questions  with  the  instructor;  (b)  answering  questions  from  the  instructor,  asking  questions  with  learners 
from  a  different  site;  (c)  answering  questions  from  learners  at  a  different  site;  (d)  asking  questions  with 
learners  from  the  same  site;  and  (e)  answering  questions  from  learners  at  the  same  site. 

A  Cronbach's  Coefficient  Alpha  of  .89  was  calculated  for  the  overall  reliability  of  the  survey.  For 
the  subsection  Asking  Questions  a  reliability  of  .72  and  for  Answering  Questions  a  reliability  of  .75  were 
calculated. 


Results  and  Discussion 

Do  learners  at  the  studio  site  and  the  remote  sites  perceive  it  easier  to  ask  and  answer  questions  in 
the  traditional  class  or  in  the  TV  class?  (RQ1)  Single  sample  t-tests  were  conducted  for  both  studio  and 
remote  groups  on  the  dependent  variable  perceived  ease  of  sending  messages.  The  Studio  site  {n  =  97] i  had  a 
mean  of  -.02  (SD  .83,  t  =  -.24).  The  Remote  sits  (n  =  121)  yielded  a  mean  of  .19  (SD  =  .64,  t  =  3.34). 
Table  1  reports  the  results  of  the  t-tests  for  both  the  studio  and  the  remote  groups. 

Table  1 .  Descriptive  Statistics  for  the  Single  Sample  T-Test 

Site  N  Mean  SD  I  

"sSdto                             97                    Tb!  _ "      .83  -24 
Remote  121  J9  -M  33£  


*  p<.01 


As  shown  in  Figure  1,  a  mean  =  0  would  indicate  that  the  perceived  ease  of  sending  messages  was 
the  same  for  the  traditional  and  the  TV  methods  of  instruction.  A  positive  mean  would  indicate  sending 
messages  in  the  TV  class  was  perceived  to  be  easier,  and  a  negative  mean  would  indicate  the  traditional  class 
was  perceived  as  easier  for  sending  messages.  Learners  from  the  remote  site  locations  rated  the  TV 
classroom  as  significantly  easier  for  asking  and  answering  questions  than  the  traditional  setting.  Although 
significant,  the  effect  size  was  relatively  low,  Cohen's  d  =  .29.  Learners  from  the  studio  class  did  not  rate 
the  two  as  significantly  different. 


Figure  1.  TV  vs.  Traditional  in  a  Single  Sample  T-Test 
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For  the  studio  group,  the  most  obvious  differences  between  the  TV  and  the  traditional  settings  is 
the  presence  of  the  technical  equipment  in  the  TV  class.  This  equipment  does  not  seem  to  make  the  TV 
class  different  in  terms  of  the  perceived  ease  of  sending  messages.  For  the  remote  group,  the  differences 
include  the  technology  and  the  absence  of  the  instructor.  Perhaps  not  having  the  instructor  in  the  same 
room  creates  a  less  intimidating  situation  where  the  learner  feels  it  easier  to  send  messages.  This  may  be 
particularly  true  with  the  population  unique  to  Hawaii  that  is  composed  of  many  cultures,  predominantly 
Asian.  With  a  small  effect  size  and  an  atypical  population,  caution  should  be  observed  when  extrapolating 
these  results.  Further  research  is  needed  to  confirm  these  results  with  other  cultural  groups. 

When  interacting  with  the  instructor,  does  the  learner's  origination  site  (studio  or  remote)  affect  the 
perceived  ease  of  asking  and  answering  questions?  (RQ2)  A  2x2  ANOVA  with  repeated  measures  on  the 
independent  variable  message  type  was  conducted.  The  independent  variable  message  type  had  two  levels: 
ask  and  answer.  The  independent  variable  sender  location  had  two  levels:  studio  and  remote.  (See  Figure 
2.)  The  learner  location  (studio  or  remote)  or  type  of  communication  (ask  or  answer)  did  not  have  a 
significant  effect  on  learner  perceptions  of  the  ease  of  communication  with  the  instructor.  These  results 
indicate  being  physically  separated  from  the  instructor  does  not  affect  how  easy  learners  feel  it  is  send 
questions  and  answers.  Perhaps  the  instructors  in  these  courses  have  been  able  to  successfully  use  strategies 
that  facilitate  learner-instructor  interaction 
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Figure  2.  Communication  with  the  Instructor  using  2-Way  ANOVA  Repeated  Measures. 
3.6 


_B —  Ask 

J —  Answer 


Perceived  Ease 


Studio 
n  =  97 


Remote 
n=  121 


When  interacting  with  the  other  learners,  does  the  learner's  origination  site  (studio  or  remote)  and 
the  destination  site  (same  or  different)  affect  the  perceived  ease  of  asking  and  answering  questions?  (RQ3)  A 
2x2x2  ANOVA  with  repeated  measures  on  the  independent  variables,  message  type  and  destination,  was 
conducted.  The  first  independent  variable,  message  type,  had  two  levels:  ask  and  answer.  The  second 
independent  variable,  sender  location,  had  two  levels:  studio  and  remote.  The  third  independent  variable, 
destination,  had  two  levels:  same  site  and  different  site.  Table  2  shows  the  means  of  the  different  groups 
and  Table  3  shows  the  results  from  the  ANOVA.  The  main  effect  learner  location  was  significant  (rf/=  ' .  F 
=  6  07  p<  05)  Learners  at  the  remote  site  perceived  asking  and  answering  questions  with  other  learners  as 
significantly  easier  than  the  learners  at  the  studio  sites.  Again,  not  having  the  instructor  physically  present 
in  the  classroom  may  have  a  relaxing  effect.  The  learners  may  feel  they  can  more  easily  interact  with  other 
learners  without  disturbing  the  lecture. 


Table  2.  Descriptive  Statistics  for  3-Way  ANOVA 

Ask  Answer 
Different  Same  Different  Same 

Mean         SD  Mean  SD  Mean          SD         Mean.  SD_ 

TWte-  TTl  L40         197  L05  ~94 3^T  97 

Studio  4.55  1-38  3.36  1.19  4^3  \J0  3,52  L25_ 


Table  3.  Learner  to  Learner  Communication  using  a  3-Way  ANOVA  with  Location,  Message  and 
Destination 


Source 


Location 

Message 

Destination 

LocatXMess 

DestXLocat 

DestXMess 

DXMXL 


SS 


df 


MS 


13.62 
.36 
238.72 
.21 
1.52 
15.91 
.13 


13.62 
.36 
238.72 
.21 
1.52 
15.91 
.13 


6.07** 
.43 
136.94* 
.26 
.87 
25.55* 
.64 


*  p<.0\  **p<.05 
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The  main  effect  of  message  was  not  significant.  Overall  learners  perceived  no  difference  between 
the  ease  of  asking  and  answering  questions.  However,  the  main  effect  of  destination  was  significant  (df=  1 , 
F  =  1 36.94,  p<.01 ).  Learners  perceived  it  to  be  significantly  easier  to  ask  or  answer  questions  at  the  same 
site  rather  than  at  the  a  different  site.  These  results  suggest  the  importance  of  the  physical  presence  of  the 
other  learner  in  learner-learner  interaction.  Learners  have  limited  contact  with  other  students  at  different 
sites.  Different  site  students  are  not  viewed  on  the  screen  very  often,  and  usually  the  whole  class  will  be 
shown  from  a  distance  rather  than  close-ups  of  each  individual.  Interaction  with  other  learners  at  the  same 
site  can  be  informal  and  does  not  require  the  use  of  technology.  In  order  to  interact  with  a  learner  at  a 
different  site,  the  microphones,  TV's,  and  cameras  must  be  used. 

The  interaction  effect,  destination  by  message  type,  was  significant  (df=  1,  F  =  25.55,  p<.05). 
(See  Figure  3  and  4.)  Learners  perceive  sending  answers  to  be  easier  than  asking  questions  when 
communicating  with  learners  at  a  different  site;  however,  they  found  asking  questions  to  be  easier  than 
answering  questions  when  communicating  with  learners  at  the  same  site.  When  communicating  in  the  TV 
classroom,  directing  a  question  to  a  learner  at  a  different  site  is  perceived  as  harder  than  responding  to  a 
question  posed  by  a  learner  at  a  different  site.  Perhaps  answering  is  easier  because  a  response  has  been 
prompted,  while  asking  a  question  requires  initiating  a  communication  behavior  with  a  physically  and 
possibly  psychologically  distant  individual.  If  learner  to  learner  interaction  is  more  informal  and  occurring 
while  the  instructor  is  lecturing,  then  answering  a  question  from  a  learner  at  the  same  site  may  mean 
missing  transient  information.  The  learner  asking  the  question  is  willing  to  miss  information,  the 
individual  being  asked  may  not  want  to  divert  attention  away  from  the  instructor. 

The  interaction  effects  of  location  by  message  and  destination  and  location  were  not  significant. 
Learners  at  both  studio  and  remote  sites  perceive  asking  and  answering  questions  with  the  same  relative 
ease.  Also,  as  would  be  expected,  learners  at  both  studio  and  remote  sites  perceive  it  easier  to  ask  or  answer 
questions  within  the  same  site  rather  than  a  different  site.  The  three-way  interaction  effect  destination  by 
message  type  by  learner  location  was  not  significant.  The  pattern  of  the  destination  by  the  message  type 
interaction  is  the  same  for  both  the  studio  and  remote  sites.  This  is  shown  graphically  by  comparing  of 
Figure  3  and  Figure  4. 
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Figure  4.  Learners  at  the  Studio  Site  with  Same  Site  and  Different  Site  Learners  using  a  3-Way  ANOVA 
with  repeated  measures  on  Message  and  Destination. 


Conclusions 

The  results  of  this  research  indicate  certain  attributes  of  the  interactive  television  classroom  do 
significantly  affect  the  learner's  perceived  ease  of  asking  and  answering  questions.  Instructors  should  be 
aware  of  these  effects  when  planning  courses  to  be  taught  over  the  television.  Based  on  the  results  of  this 
study,  future  research  can  be  directed  into  studies  on  other  effects  of  the  TV  medium  on  learning  processes. 
Achievement  in  the  traditional  and  television  classrooms  may  be  the  same,  but  other  differences  between  the 
two  can  still  be  found. 

The  media  characteristics  of  two-way  television  seem  to  create  greater  barriers  between  learners 
across  sites  than  between  learners  and  the  instructor  or  learners  at  their  own  site.  The  design  of  the  system 
commonly  disallows  visual  contact  between  learners  at  the  various  sites.  TV  monitors  for  all  sites  are 
often  only  present  at  the  studio  site,  and  depending  on  the  set-up  may  not  be  easily  visible  even  by  the 
studio  learners.  Visual  contact  is  dependent  on  the  instructor  or  technician  choosing  to  show  an  individual, 
group  of  students,  or  site  on  the  program  monitor.  In  an  interview  with  students  at  a  remote  site,  students 
said  that  it  was  easier  to  interact  with  the  instructor  than  with  the  other  sites  simply  because  you  can  see 
them.  However,  when  other  sites  were  put  on  the  screen,  interaction  was  easier.  To  improve  results  with 
student  centered  learning  and  increase  learner  to  learner  interaction,  future  research  should  examine  system 
features  that  reduce  visual  barrier?  between  students. 

Since  improving  system  design  can  be  costly,  instructional  design  may  be  able  to  provide  a  more 
immediate  solution.  Instructors  should  work  towards  improving  strategies  to  increase  the  visibility  of 
individual  students  earlier  in  their  courses.  Learner  to  learner  activities  across  sites  may  increase  the 
students'  connection  to  each  other.  Electronic  mail  and  other  personal  communications  could  serve  as  a 
vital  link  for  collaborative  projects.  Students  should  be  encouraged  to  share  their  thoughts  with  others  in 
the  system  rather  than  depending  only  on  their  site  mates  for  interaction.  Research  into  the  changing 
culture  of  the  classroom  that  examines  the  "TV  room"  atmosphere  of  the  distance  classroom  may  be 
important  to  fully  understand  the  interactive  behavior  of  learners. 
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Abstract 

Satisfaction  is  a  construct  that  is  important  to  the  development  of  intrinsic  motivation  and  the 
continuing  effort  to  learn.  Research  that  helps  to  identify  those  factors  that  contribute  to  satisfaction  is 
useful  in  the  design  of  electronic  support  systems  for  individuals  and  groups.  This  paper  investigates  the 
impact  of  need  for  closure  on  information  processing  and  decision  confidence  (as  signified  by  closure)  and 
the  subsequent  relationship  between  confidence  and  satisfaction  with  the  outcomes  of  a  group  decision- 
making task  .This  paper  presents  a  brief  review  of  the  literature  related  to  closure  and  satisfaction  then 
describes  the  Cogmt.ve-Motivational  Model  of  Group  Member  Decision  Satisfaction,  and  concludes  with 
the  results  of  an  initial  empirical  research  study  of  this  model  with  groups  using  either  a  computer-based 
group  system  or  a  manual  method  for  supporting  group  decision-making. 

Introduction 

Groupwork  has  become  an  important  focus  of  research  in  education,  as  evidenced  by  the  growine 
amount  of  research  on  cooperative  learning  in  all  educational  contexts  (e.g.  Johnson,  Johnson,  &  Stanne 
1985,  Hooper,  1994;  Hooper  &  Hannafin,  1991).  Research  indicates  that  students  working  in  cooperative 
groups  are  more  successful  at  high  level  skills  such  as  problem  solving  and  decision-making  than  when 
working  alone.  Group  learning  situations  may  include  such  experiences  as  simulations,  debates  group 
projects,  and  case  study  analyses.  In  such  situations,  a  group  is  often  faced  with  a  problem  or  set  of 
problems  and  the  task  involves  reaching  consensus  and  forming  a  group  judgment  or  decision. 

Although  there  has  been  considerable  attention  in  the  education  literature  to  the  effects  of  group 
learning  on  achievement  (e.g.  Johnson  &  Johnson,  1989;  Slavin,  1990;  Hooper  &  Hannafin  1991)  until 
recently  there  has  been  considerably  less  attention  paid  to  affective  outcomes,  such  as  satisfaction  with 
group  work,  specifically  group  decision  making.  Yet,  when  assessing  a  decision,  group  member  satisfaction 
is  a  critical  variable  and  may  play  a  significant  role  in  decision  implementation  (Maier,  1970)  and  srouo 
effectiveness  (e.g.  Van  de  Ven  &  Delbecq,  1974).  /<uiugiuup 

Despite  being  a  frequently-used  research  variable,  satisfaction  has  not  been  adequately 
conceptualized  or  defined  (Hecht,  1978).  When  people  are  asked  if  they  are  satisfied  wim  a  particular  activity 
or  outcome  and  they  say  "yes,"  what  does  that  mean?  Does  it  mean  that  the  task  was  enjoyable,  rewarding 
and  or  important?  Does  it  mean  that  they  would  do  more  of  it  or  do  it  again?  Thus,  research  on  group 
member  satisfaction  may  serve  two  purposes.  From  a  theory-building  standpoint,  satisfaction  can  serve  as  a 
criterion  for  research  evaluating  other  variables  (Smith,  et  al.,  1966).  From  a  practical  standpoint  the  study 
ot  satisfaction  may  have  direct  implications  for  the  conduct  of  group-based  tasks  like  group  decision- 
making. 

Our  theory  of  group  member  satisfaction  is  based  on  Kruglanski's  (1 989)  cognitive-motivational 
theory  of  closure,  where  closure  is  defined  as  subjective  certainty,  reflecting  a  state  of  firm  knowledge  This 
state  once  attained,  signals  a  degree  of  closed-mindedness,  i.e.  the  point  at  which  new  information  bearing 
on  the  decision  problem  will  not  be  actively  sought  by  the  decision-maker  (Kruglanski,  1989)  As  a  result 
we  reason  that  the  attainment  of  closure  will  be  an  important  component  of  group  member  satisfaction  with 
the  decision  and  we  have  developed  a  model  based  on  that  assertion.  This  paper  reviews  some  of  the  related 
hterature,  builds  on  our  Cognitive-Motivational  Model  of  Group  Member  Decision  Satisfaction  [formerly 
Closure  Model  of  Decision  Satisfaction  (Small  &  Venkatesh,  1994)]  and  presents  the  results  of  an  initial 
research  study  testing  this  model. 

Closure 

The  Random  House  Dictionary  of  the  English  Language  ( 1 987)  defines  closure  in  terms  of 
psychological  completeness  or  certainty].  Closure  has  been  a  topic  of  study  in  at  least  three  areas- 
education,  management,  and  psychology. 

Educational  researchers  have  tended  to  focus  on  the  former  component  without  defining  "closure- 
in  operational  terms.  The  importance  of  closure  in  accomplishing  a  learning  task  has  been  recognized 
frequently  in  the  education  literature.  Dubelle  (1986)  describes  closure  in  terms  of  outcomes  of  an  activity 
that  bring  the  major  points  of  a  lesson  into  focus  so  they  may  be  perceived  as  an  organized  whole.  Keller 
(1983)  emphasizes  a  learner's  need  to  perceive  various  pieces  of  content  as  fitting  into  a  whole  thereby 
experiencing  closure  and  a  sense  of  accomplishment.  Ziergarnik  (1967)  emphasizes  the  need  to  bring  an 
instructional  task  to  closure  and  Brophy  (1987)  contends  that  students  experience  a  sense  of  accomplishment 
when  they  complete  meaningful  tasks.  Wlodkowski  (1991)  advocates  attaining  "positive  closure"  as  a 
motivational  elemenl  in  adult  education  "because  it  affirms  the  entire  learning  process,  Mnforces  the  value 
of  the  experience,  directly  or  indirectly  acknowledges  competence,  increases  cohesiveness  within  the  group 
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and  encourages  the  sui facing  of  inspiration  and  other  beneficial  emotions  within  the  learners  themselves" 
(pn  247-248)  They  all  refer  to  closure  as  an  end  state  that  reflects  the  accomplishment  of  a  task. 

Closure  has  also  been  addressed  in  the  management  literature  on  group  research  but  no-one  has 
clearly  defined  it  In  a  study  comparing  interacting  nominal  group  technique  (NGT)  and  delphi  groups,  Van 
de  Ven  and  Delbecq  (1974)  found  NGT  groups  attained  "closure,"  while  delphi  groups  attained  closure  with 
detachment"  and  interactive  group  meetings  tended  to  conclude  with  "high  perceived  lack  of  closure. ' 
Notably  NGT  members  were  also  more  satisfied  with  group  process  and  performance,  suggesting  a  posit.ve 
link  between  closure  and  satisfaction.  The  authors,  however,  fail  to  make  such  a  link  and  never  define 
closure,  conceptually  or  operationally. 

Hagen  &  Burch  (1985)  found  that  the  perception  of  "closure  on  a  group  task  was  positively 
related  to  member  satisfaction.  However,  they,  too,  do  not  not  define  closure.  It  is  also  unclear  whether 
satisfaction  was  due  to  process  factors  and/or  task  outcomes.  Bostrom  et  al.  (1993)  identify  lack  of  closure 
as  one  of  the  major  problems  behind  unproductive  meetings;  "lack  of  closure"  here  describes  meetings  that 
typically  generate  no  published  outputs,  run  too  long,  or  are  generally  inconclusive.  Anderson  &  Roberta 
(1985)  see  the  human  facilitator  as  promoting  "closure"  through  a  mediated  resolution  of  issues.  While 
these  references  are  intriguing,  no  operational  definition  of  closure  is  advanced. 

Closure  has  been  explored  more  thoroughly  in  the  psychology  literature,  most  notably  by 
Krualanski  Kruglanski  (1989)  incorporates  "closure"  into  a  cognitive-motivational  theory  of  knowledge 
and  cognition  in  which  a  "need  for  closure"  is  assumed  to  motivate  a  person  to  prefer  certainty  over 
indeterminacy.  Need  for  closure,  largely  a  situation  variable,  is  imposed  by  factors  in  the  environment, 
such  as  time  constraints,  a  desire  to  move  on  to  other  matters,  or  the  fear  of  an  inaccurate  decision. 

Closure  is  an  end  state,  characterized  by  confidence  in  the  possession  of  a  definite  unambiguous 
answer  to  a  question,  "as  opposed  to  confusion  or  ambiguity"  (Kruglanski  &  Freund,  1983,  p  450).  While 
a  person's  need  for  closure  may  be  characterized,  by  high  (  hereafter  referred  to  as  need  to  expedite)  closure  or 
low  (hereafter  referred  to  as  need  to  delay)  closure,  the  underlying  concept  reflects  a  quest  for  assured 
ESL  affords  predictability  and  a  base  for  action"  (Kruglanski,  1989,  p.  14)  Closure  theory  posits 
that  information  processing  in  judgmental  situations  is  influenced  by  the  dec.s.on-mak  _er  s  "^J for  closure 
as  specified  on  a  continuum  of  openness  to  new  information  (or  degree  of  close-mindedness)  (Kruglanski, 
1989)  Although  possibly  influenced  by  individual  differences  variables  (e.g.  tolerance  for  ambiguity,  trait 
curiosityX  need  for  closure  is  largely  a  situation-specific  variable  whereby  costs  and  benefits  are  assumed  to 
affect  level  of  need  for  closure  when  the.  individual's  current  state  differs  from  a  desired  end  state.  Therefore, 
the  individual  must  perceive  closure  as  a  current  benefit  in  order  to  have  a  need  to  expedite  closure  sooner 
rather  than  later.  However,  the  theory  does  not  consider  the  affective  consequences  of  either  attaining  ot 

attaining  closur^  ^  ^  allowing  additional  information  to  be  considered,  and  keeping  an 

open  mind  before  commiting  to  a  decision  or  course  of  action  because  the  perceived  benefits  of  delaying 
closure  outweigh  the  costs  of  lacking  closure,  need  to  delay  closure  tends  to  emphasize  accuracy  (requiring 
more  time  on  Mfck),  while  need  to  expedite  closure  tends  to  emphasize  other  goals  (e.g  the  need  to  sec 
oneself  as  decisive),  with  accuracy  assuming  a  less  critical  role.  For  example,  when  a  decision  or  action  is 
consequential  (e.g.  an  actual  courtroom  murder  trial),  it  is  reasonable  to  assume  that  the  decision-maker 
( iuror)  given  a  choice,  would  delay  making  a  decision  in  order  to  process  information  more  extensively 
before  rendering  a  judgment,  given  the  high  cost  of  judgmental  error  inherent  in  the  situation  (convicting  an 
innocent  person).  A  low  need  to  attuln  closure  tends  to  emphasize  accuracy  to  avoid  the  high  cost  of  error. 
However  even  in  need  to  delay  closure  situations,  as  in  the  above  example,  an  individua  will  be  motivated 
to  eventually  attain  closure.  Only  under  certain  circumstances  will  decisions  be  avoided  altogether  (Corbin, 

1980;  Mann  &  Janis,  1982).  k»™,«:„tw„ 
In  a  state  of  need  to  expedite  closure,  the  decision-maker  wishes  to  expedite  closure  because  the 
perceived  benefits  outweigh  the  costs  of  lacking  closure.  For  example,  when  a  decision  is  relatively  ess 
consequential  (e.g.  a  mock  trial),  the  decision-maker  (e.g.  student  juror)  may  be  more  willing  to  render  a 
qrick  judgment  where  the  perceived  benefits  of  closure  (e.g.  more  time  to  do  other  things)  outweigh  the 
-Ueived  costs  of  lacking  closure  (e.g.  the  need  to  invest  additional  information  processing  time  and  effort). 
High  need  to  attain  closure  is  strongly  influenced  by  such  factors  as  self-esteem  or  cognitive  consistency,  as 
well  as  the  need  for  accuracy  (Pyszczynski  &  Greenberg.  1987).  .  _ 

The  need  for  closure  also  influences  the  information  processing  component  of  the  decision-making 
process  (Guzzo  1982).  Uncertainty  reduction  stimulates  information  seeking  and  maintains  information 
acquisition  behavior  (Lanzetta,  1963).  When  an  individual  understands  that  he  has  an  'information  gap 
between  what  he  knows  and  what  he  wants  or  needs  to  know,  he  will  be  motivated  to  seek  out  the 
information  he  needs  to  close  that  gap  (Lanzetta,  1963). 


In  decis.on-making  s.tuat.ons,  information  processing  includes  both  hypothesis  generation  (\  e 
possible  solutions  based  on  the  information  received)  and  hypothesis  evaluation  [i.e.  selection  of  the  most 
e  ficac.ous  so  ution(s)]  (Gettys,  Mehle,  and  Fisher,  1986;  Kruglanski,  1989).  Mayseless  &  Kruglanski 
1987)  found  less  extensive  information  processing  was  needed  to  produce  a  confident  judgment  under  need 
o  expedite  closure  than  need  to  delay  closure  conditions  (which  was  experimentally  induced  by  heightening 
the  fear  of  invalidity  and,  hence,  the  need  for  judgmental  accuracy).  In  the  need  to  delay  closure  condition 
more  extensive  information  processing  was  required  for  subjects  to  feel  confident  enough  to  render  a  ' 
judgment  believed  to  be  accurate. 

Closure  and  Affect 

We  extend  Kruglanski's  (1989)  closure  theory  to  include  affect.  That  is,  to  the  extent  that  a 
decision-maker  is  motivated  to  close  the  gap  between  a  current  state  of  lack  of  closure  and  a  desired  end  state 
or  closure,  we  posit  that  attaining  that  desired  end  state  will  promote  positive  affect,  while  failure  to  attain 
it  will  cause  negative  affect,  in  both  high  and  need  to  delay  closure  conditions.  In  general  terms  group 
member  satisfaction  is  an  affective  resoonse  to  some  type  of  cognitive  appraisal  of  accomplishment  of  a 
group  task,  process,  composition,  or  decision  (Witteman,  1991;  Collins  *  Guetzkow,  1964)  Researchers 
have  found  group  member  satisfaction  highly  correlated  with  degree  of  task  accomplishment  (c  g  Heslin  & 
Dunphy  1960;  Marquis,  Guetzkow  &  Heyns,  1951),  the  context  on  which  our  research  focuses.  However 
it  is  unclear  from  the  literature  what  the  appraisal  process  involves  (i.e.  how  one  decides  that  the  task  has  ' 
been  adequately  accomplished)  or  how  cognitive  appraisal  (i.e.  closure)  and  the  affective  response  are  linked 
(Hecht,  1978). 

Our  theory  represents  the  decision-maker  as  an  active  evaluator  of  the  adequacy  of  information 
processing,  using  what  Corbin  (1980)  calls  the  "uncertainty  cutoff  as  the  subjective  standard  of  appraisal 
We  assert  that  information  processing  will  continue  until  the  decision-maker  has  attained  the  required  level 
of  confidence  with  the  knowledge  he/she  possesses  at  that  point  (a  subjective  assessment)  and  decides  to 
decide.  Closure  theory  posits  that  information  processing  in  judgment  situations  is  influenced  by  the 
decision-maker's  motivation  (need  for  closure)  as  specified  on  a  continuum  from  low  to  high  (Kruglanski 
1989).  When  this  state  of  closure  (or  decision  confidence)  is  reached,  the  decision-maker  will  halt 
information  processing  (albeit  temporarily)  and  make  a  decision.  Thus,  satisfaction  with  the  decision  at  the 
time  the  decision  is  made  is  the  affective  outcome  of  having  attained  the  desired  cognitive  end  state  of 
•  closure. 

Closure  should  be  a  stronger  predictor  of  decision  satisfaction  in  need  to  delay  closure  conditions 
than  in  need  to  expedite  conditions.  That  is,  the  strength  of  the  relationship  between  closure  and  decision 
satisfaction  will  vary  depending  on  whether  epistemic  motivations  emphasize  accuracy  or  other  goals  When 
accuracy  is  emphasized,  confidence  (resulting  in  closure)  will  be  a  strong  predictor  of  decision  satisfaction 
reflecting  high  subjective  value  placed  on  confidence  in  such  situations.  When  other  goals  dominate, 
decision  confidence  will  be  a  relatively  weaker  predictor  of  decision  satisfaction. 

Group  Member  Satisfaction 

Porter  &  Lawler  (1968)  found  a  relationship  between  performance  and  satisfaction  in  the 
workplace,  where  the  success  of  the  former  is  a  cause  of  the  latter.  In  learning  situations,  satisfaction  is 
often  described  as  the  sense  of  accomplishment  that  learners  feel  at  the  conclusion  of  a  learning  event  when 
outcomes  of  their  efforts  are  consistent  with  their  expectations  (e.g.  Keller,  1983). 

Typically  satisfaction  is  defined  and  measured  in  overly  simplistic,  unidimensional  terms  (e  "  Was 
it  fun?  Are  you  satisfied?)  Klein  &  Pridemore  (1992)  measured  satisfaction  using  six  related  items  from  the' 
Instructional  Materials  Motivation  Scale  (Keller,  1987).  In  addition  to  using  enjoyment  and  feeling  good 
md  pleasure  as  descriptors,  the  items  descril  ed  satisfaction  in  terms  of  "accomplishment,"  and  "practice  and 
feedback." 

Some  current  research  points  to  a  relationship  between  satisfaction  and  other  constructs.  For 
example,  Small  and  Gluck  (1994)  used  a  magnitude  scaling  method  to  investigate  adult  student  perceptions 
of  the  relationships  of  35  instructional  attribute  terms  to  four  major  motivational  components  (attention, 
relevance,  confidence,  and  satisfaction)  comprising  Kellers  ARCS  Model  of  Motivational  Design  (e.g. 
Keller,  1983),  a  model  largely  based  on  expectancy-value  theory  (Porter  &  Lawler,  1968).  Their  results 
found  that  feedback  and  related  terms  (encouragement,  praise,  supportiveness)  were  among  the  most  closely 
associated  to  satisfaction.  In  addition,  they  explored  relationships  among  the  four  ARCS  components.  They 
found  that  confidence  and  satisfaction  shared  seven  of  the  ten  closest  attributes.  They  also  found  that  the 
attributes  related  to  confidence  and  satisfaction  were  significantly  different  than  those  related  to  attention  and 
relevance.  They  point  to  this  as  evidence  supporting  Keller's  claim  that  the  confidence  and  satisfaction 


conditions  are  closely  related  to  each  other,  as  well  as  with  the  "expectancy  for  success"  factor  of 
expectancy-value  theory,  while  attention  and  relevance  conditions  are  aligned  with  the  "value"  factor. 

Conceptually,  satisfaction  derives  from  an  affective  or  emotional  response  to  a  situation.  Arnold 
(1960)  in  a  now  widely-accepted  characterization,  saw  satisfaction  as  an  outcome  of  an  appraisal  process, 
with  situations  "seen  as  favorable  to  one's  well-being...appraised  positively,  while  those  seen  as 
inimical-appraised  negatively"  (Locke  &  Latham,  1990,  p.  226).  A  performance  goal  may  serve  as  the 
value  standard  used  to  assess  performance,  with  positive  affect  resulting  if  the  standard  is  exceeded  and 
negative  affect  when  the  standard  is  not  met  (Locke,  1976). 

Often  the  focus  of  research  is  on  the  individual  learner's  satisfaction  following  successful 
completion  of  a  task.  However,  when  learners  are  required  to  work  in  groups,  use  group  processes, 
accomplish  group  goals,  and  be  assessed  according  to  group  outcomes,  group  member  i  tisfaction  may  not 
reflect  individual  satisfaction;  i.e.  an  individual  could  be  satisfied  that  the  group  reached  a  group-level  goal 
but  is  personally  dissatisfied  with  the  result  or  process  (Small  &  Venkatesh,  1994).  This  research  examines 
individual  satisfaction  with  accomplishment  of  a  group  task. 


The  Model  ,  .    .  . 

Our  work  extends  Kruglanski's  (1989)  cognitive-motivational  perspective  to  examine  decision 
satisfaction  as  researchers  have  stressed  the  value  of  studying  satisfaction  (e.g.  Smith  et  al.,  1957)  and 
derision-making  (Corbin,  1980)  in  motivational  terms.  The  cognitive-moti vatic  A  perspective  on  closure 
centers  on  information  processing  effects.  As  information  processing  is  central  to  group  decision-makir.g 
(Guzzo,  1982)  and  GSS  environments  (DeSanctis  &  Gallupe,  1987),  we  reasoned  that  closure,  as  a  vamble 
associated  with  information  processing,  would  predict  decision  satisfaction. 

The  Cognitive-Motivational  Model  of  Group  Member  Decision  Satisfaction  (presented  in  Hgure 
1)  was  first  developed  at  the  individual  level  and  then  considered  in  the  group  environment  (Small  & 
Venkatesh  1994)  This  model  provides  a  theory-baseci  linking  of  motivation,  information  processing, 
confidence  (signifying  closure)  and  satisfaction.  This  model  is  based  on  the  following  predictions  regarding 
decision  satisfaction:  (1)  if  closure  is  desired  or  valued  (e.g.  the  cost  of  judgmental  invalidity  is  low  and/or 
the  benefit  of  rendering  an  expeditious  judgment  is  high),  rendering  such  a  judgment  will  promote  positive 
affect  in  the  decision-maker  and  conversely  (2)  in  situations  where  closure  is  not  desired  or  valued  (the  cost 
of  judgmental  invalidity  is  high  so  the  decision-maker  has  a  compelling  incentive  to  "keep  an  open  mind 
and  delay  or  postpone  making  a  decision),  the  decision-maker  will  experience  negative  afiect  if  forced  to 
render  a  judgment  (e.g.  due  to  imposed  or  perceived  time  constraints).  In  motivational  terms,  the  first 
situation  signifies  a  need  to  expedite  closure  (confidence)  (Kruglanski,  1989).  The  Cognitive-MoUvat.onal 
Model  of  Group  Member  Decision  Satisfaction  posits  a  number  of  paired  relationships  including  need  tor 
closure  and  extent  of  information  processing,  information  processing  and  confidence  (signifying  closure), 
and  confidence  and  satisfaction.  Each  are  explained  briefly  beiow. 


Figure  1.  The  Cognitive-Motivational  Model  of  Group  Member  Decision 
Satisfaction 


Need  for  Closure  and  Extent  of  Information  Processing.  Need  for  closure  (high  versus  low) 
Lengr,,el?e  CXtent  of  info™"ion  processing  (Kruglanski,  1989);  i.e.  in  general,  information  processing 
tends  to  be  more  extensive  (breadth)  as  well  as  more  intensive  (depth)  in  low  (versus  high)  need  for  closure 
conditions  (Mayseless  &  Kruglanski,  1987).  Lanzeita  (1963)  states  that  under  conditions  of  heightened 
uncertainty  active  information  acquisition  and  processing  is  more  probable.  He  found  that  for  any  level  of 
uncertainty,  information  seeking  will  be  greater  the  more  important  the  consequences  of  the  decision  Let  us 
consider  the  individual  situations  of  deciding  when  to  submit  a  course  term  paper  and  when  to  submit  a 
doctoral  o.ssertation.  hor  the  former,  need  to  expedite  closure  will  likely  be  high,  due  to  perceptions  of 
possible  external  factors  such  as  time  constraints  (e.g.  inflexible  due  date),  lower  comprehensiveness 
requirement  (e.g.  10-page  maximum),  or  less  emphasis  on  quality  (e.g.  only  one  of  several  assignments 
considered  for  a  grade).  However,  for  the  latter  the  need  to  delay  closure  may  be  quite  high  and  information 
processing  will  likely  continue  for  a  much  longer  period  of  time  due  to  the  high  cost  of  error  (e  g 
submitting  an  inadequate  literature  review;  insufficient  statistical  analysis) 

Information  Processing  and  Confidence,  a  key  outcomes  in  decision  making,  signifies 

certa.ntv  (Sn.ezek,  1992).  Information  processing  and  confidence  are  reciprocally  linked;  i  e  information 
processing  is  undertaken  to  reduce  the  level  of  subjective  uncertainty  inherent  in  a  decision  until  the 

,™'y  CUt°r  15  reaChed'  dedsi0n  confidence  is  high'  ^d  the  -timing  of  choice"  is  at  hand  (Corbin 
1980).  This  cutotf  is  subjectively  derined  with  reference  to  a  level  of  confidence  that  the  decision-maker 
deems  acceptable.  In  the  above  example,  the  uncertainty  cutoff  is  most  likely  to  be  reached  much  earlier  for 
the  term  paper  than  for  the  dissertation.  If,  however,  confidence  is  below  the  uncertainty  cutoff  the 
decision-maker  may  be  motivated  to  avoid  or  delay  a  decision,  which  could  happen  in  either  of  the  above 
examples.  Using  the  uncertainty  cutoff  idea,  our  model  augments  Kruglanski  by  specifying  the  motivation 
behind  the  decision  to  halt  (albeit  temporary)  information  processing  and  the  decision  to  decide 

But,  one  might  wonder,  what  constitutes  an  acceptable  level  of  confidence  and  by  what  criterion  is 
the  uncertainty  cutoff  subjectively  defined?  A  "cost-benefit"  analysis  of  the  effects  of  difference  need  for 
closure  motivations  on  the  extent  of  information  processing  suggests  that,  for  need  to  expedite  conditions 
reducing  the  cost  (to  the  decision-maker)  of  judgmental  invalidity  and/or  increasing  the  benefit  of  rendering 
an  expeditious  judgment  will  limit  information  processing.  Conversely,  in  need  to  delay  closure  conditions 
increasing  the  cost  of  judgmental  invalidity  fosters  extensive  information  processing  (Kruglanski  1989) 

Intensive  processing  of  available  information  or  gathering  more  information  provide  an  alternative 
means  for  decreasing  uncertainty  and  for  inducing  the  readiness  to  decide"  (Corbin,  1980,  p.  54)  Need  for 
closure  will  regulate  information  processing  until  the  subjectively-set  uncertainty  cutoff  is  exceeded  (or 
some  f  .ternal  force  intercedes),  at  which  point  it  will  be  halted  (albeit  temporariW)  and  a  decision  made 
This  perspective  implies  a  decision-maker  who  actively  assesses  the  costs  versus  benefits  of  processing 
information  in  light  of  a  subjective  need  for  closure.  Arguably,  then,  information  processing  that  is  judged 
as  adequate  under  need  to  expedite  conditions  may  be  judged  as  inadequate  under  low  need  for  closure  ° 
conditions.  The  student  writing  the  term  paper  (need  to  expedite)  may  decide  to  stop  processing  information 
and  complete  the  task  by  the  due  date  in  order  to  avoid  getting  a  lower  grade  for  handing  in  the  paper  late 
The  student  writing  the  dissertation  (need  to  delay  closure)  may  decide  to  delay  submission  of  a  final  product 
when  she  learns  of  an  alternative  line  of  research  from  another  discipline  that  she  has  not  included  in  her 
literature  review  and  believes  it  will  better  prepare  her  for  her  oral  defense.  The  latter  decision-maker  would 
i.ave  to  be  more  confident  than  the  former  before  choosing  to  halt  information  processing  and  make  a 
decision  to  submit  because  the  stakes  will  be  perceived  as  higher.  (It  is  assumed  that  the  individual  will 
eventually  halt  information  processing  and  make  a  decision.)  To  summarize,  it  may  be  argued  that  (1)  the 
uncertainty  cutoff  is  defined  with  reference  to  the  decision-maker's  need  for  closure  and  (2)  confidence  during 
the  adequacy  assessment  process  will  rely  on  the  uncertainty  cutoff  to  regulate  the  extent  of  information 
processing,  with  a  decision  being  made  when  confidence  exceeds  the  cutoff.  In  the  following  section  we 
briefly  consider  four  factors  that  also  appear  to  influence  confidence  in  groups-group  process,  technology 
social,  and  motivational  factors. 

Group  Process  Fac  tors.  Groups  that  process  more  information  will  likely  be  less  confident  about 
their  decision  due  to  exposure  to  multiple  member  views  (Sniezck,  1992).  This  prediction  goes  counter  to 
the  assumption  that  more  information  may  be  processed  to  boost  confidence  (e.g.  Lanzc'.ta,  1963)  This 
apparent  contradiction  may  be  due  to  a  lack  of  clarity  in  the  literature  on  the  level  at  which  confidence  is 
measured.  More  research  directly  comparing  decision  confidence  at  the  member  and  group  levels  is  needed 
\w2)      'ink  bCtWCCn  incrC3SCfi  '"formation  processing  and  decision  confidence  is  understood  (Sniezck, 

Tevhnabfiy  Factors.  Research  on  the  use  of  computer-based  group  support  systems  (G.SS) 
suggests  that  such  systems  facilitate  information  processing  and  exchange  in  group  situations  (DeSanctis  & 
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Gallupe  1987)  Group  support  systems  (GSS)  are  a  set  of  networked  electronic  tools  that  facilitate  group 
work  most  often  in  business  settings.  They  typically  include  software  that  allows  participants  to  conduc 
group  activities,  such  as  anonymous  electronic  brainstorming  and  organization  and  ranking  of  alternatives^ 
To  the  degree  that  GSS-use  increases  the  number  of  alternatives  considered  by  the  group  (Benbasat 
&  Nault,  1990),  it  could  be  argued  that  group  confidence  would  decrease  in  GSS  groups  relative  to 
unsupported  groups  (e.g.  Sniezek,  1992).  However,  a  distinction  among  types  of  GSS  is  needed  here.  OSS- 
supZ haTbee "classified  into  group  communication  support  systems  (GCSS)  and  group  decis.on  support 
sSms  (GDSS)  (Pinnsonneault  &  Kraemer,  1989).  While  the  former  support  group  communication 
pSeTi  and  have  a  negative  effect  on  decision  confidence  and  satisfaction,  the  latter  P™*^™n^n* 
techniques  intended  to  reduce  uncertainty,  resulting  in  increased  decision  confidence  and  satisfaction 
DeSanS  &  Gallupe,  1987).  Our  model  would  predict  that,  for  choice  tasks,  member  decision  confidence 
an! IS TsaUsSon  should  increase  to  the  degree  that  GSS  supports  both  hypothesis  generation  and 

validation^  ^  ^         interaction  on  group  performance  and  achievement  have  been 

widely  explored  (e.g.  Hooper  et  al.,  1994;  Hooper  &  Hannafin,  1991;  Webb,  1982)  Specifically, 
RoSaugh  (1981 ;  1979)  and  Ono  &  Davis  (1988)  found  that  group  interaction  and  discussion  may  increase 

confidence  in  groups.  The  goal  (explicit  or  implicit)  of  reaching  consensus  may  a  so  h«8hten 
group  confidence.  The  need  for  closure  may  motivate  group  members  to  strive  for  consensus 
^  consensus  may  be  functionally  equivalent  to  an  individual's  need  for  closure  (Kruglanski  &  Webster, 

1991 }'    Motivational  Factors.  Need  for  structure  may  be  functionally  similar  to  need  for  closure;  i.e.  in 
individual  decision-making,  stressing  the  value  of  order  helps  increase  decision  confidence  (Mayseless _& 
Krudanski  1987).  In  group  settings,  an  implicit  belief  appears  to  be  that  "good  process  leads  to  good 
Smes"  Sniezek  1991,  p.  149)  This  suggests  that  positive  evaluations  (e.g.  feedback)  of  the  group 
Sracuon  £o  e    m  y  positively  influence  decision  confidence  (Guzzo  et  al.  1986).  Amount  of  time  and 
effort  expanded  on  the  task  may  also  affect  confidence,  with  evaluations  bem^positive  .  people  conclude 
they  "Sd  hard  enough  (Mayseless  &  Kruglanski,  1987)  and  inflated  confidence  in  self-assessments  of 

high  closure  is  operational  as  a  cognitive  end-state  characterized  by 

the  group  member's  confidence  in  the  group  decision.  While  need  for  closure  may  encourage  close- 
mindeless  toward  new  information  on  the  topic,  Kruglanski  (1989)  and  others  (e.g.  Pyszcz.n sk,  & 
Sreenberg,  1987)  conceptualize  it  as  a  situational  index  rather  than  an  individual  difference^,  the  model 
confidence  (a  cognitive  belief)  is  shown  to  predict  decision  satisfaction  (affective  attitude)  (ftshtem,  1 966) 
AlSTthe  two  may  be  positively  correlated  (Sniezek,  1992;  Gallupe,  Desanctis  &  D.cson  1988;  Steeb 
&  Johnston  1981),  they  arVconceptually  distinct.  Satisfaction  represents  an  affective  attitude  toward  a 
decision,  while  confidence  represents  a  cognitive  confidence  as  to  the  quality  of  the  decision. 

The  Re^rCehsealchdreported  below  represents  an  initial  experiment  testing  our  model  It  examines  the 
link  between  closure  (cognitive  response)  and  decision  satisfaction  (affective  response)  by  ™n.Pulatmg 
motivation  (need  to  delay  versus  need  to  expedite)  in  a  laboratory  environment  using  technology-based 
TgSS)  and  manual  methods.  "Need  for  closure"  is  an  independent  variable  whde  "confide ace   a »  an  nd 
state  is  a  dependent  variable.  The  terms  "subjective  certainty"  and  "confidence  are  used  synonymousl, 
(Sniezek,  1991).  Information  processing  was  analyzed  on  the  basis  of  quantity  ot  ideas. 

The  following  hypotheses  were  advanced:  ,.,.„!_  , 

Hypothesis  J.      Confidence  and  satisfaction  (in  decision  making)  will  be  positively 
correlated. 

Hypothesis  2.     Information  processing  variables  (hypothesis  generation  and 
validation)  will  predict  satisfaction  through  confidence  and  not 
directly. 


This  research  constituted  a  pilot  study  for  a  subsequent  larger  study.  This  study  was  conducted  n 
two  phases  The  first  phase  involved  manual  methods;  the  second  phase  involved  VisionQuest  TM;  i.e.  a 
technology-based  group  support  system  (GSS).  As  a  result  of  the  first  phase,  some  minor  mod.f.cat.ons 
were  made  to  the  second  phase.   

Eichty-onc  undergraduate  and  graduate  students  were  subjects  enrolled  in  three  college  courses. 
Subjects  received  course  credit  for  participation.  Subjects  were  randomly  assigned  to  groups  (three  subject 
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per  group)  (technology  vs.  no  technology)  and  group  conditions  (need  to  delay  vs  need  to  expedite  closure 
Taole  1  presents  some  demographic  data  for  subject  in  this  study.  ^  )- 


Need  to  Need  to 

Delay  Expedite 

Group  Closure  Closure 

#Ss  (#Grps)  #Ss  (#Grps) 


Technology        21  (7)  21  (7) 

No  technology     18(6)  21(7) 
TOTAL  39(13)  42(14) 


M 

F 

UG 

G 

24 

18 

22 

20 

21 

18 

16 

23 

45 

36 

38 

43 

Table  1.    Need  for  Closure,  Gender,  and  Academic  Status  of  Subjects. 

„  :  „  The  ,r^tments  for  GSS  groups  were  administered  in  a  univeresity  computer  research  laboratory 
environment.  The  treatments  for  manua!  groups  were  administered  in  a  nearby  seminar  room  where  group 
members  were  seated  a,  a  conference  table.  A  different  facilitator  was  used  for  both  groups  however  Se 
facilitation  was  purely  procedural  and  tightly  scripted  to  ensure  consistency. 

Treatments 

..  Hifh  ar>d  low  evaluation  apprehension  were  used  to  operationalize  need  tc  delay  (NTD)  vs  need  to 
expedite  (NTE)  closure  respectively  (Mayseless  &  Kruglanski.  1987).  NTD  closure  subjects  were  inform^ 
that  their  prioritized  solutions  to  a  case  study  problem  would  be  evaluated  for  quality  by  a  panel  of  expert 
judges  and  tnere  would  be  penalties  for  ineffective  solutions.  This  constituted  the  high  evaluation 
apprehension  treatment  emphasizing  the  high  cost  of  judgmental  invalidity.  NTE  subjects  were  informed 
that  their  responses  would  be  reviewed  by  a  panel  of  experts  but  there  would  be  no  penalty  based  on 
solution  quality.  This  constituted  the  low  evaluation  apprehension  treatment,  encouraging  quality  work  but 
emphasizing  no  adverse  consequences  from  judgmental  invalidity.  The  treatments  were  similar  in  both 
phases  of  this  researcn. 

The  Experimental  Task 

"The  Case  of  the  Unhealthy  Hospital"  (Kovner,  1991),  acase  study  widely  used  in  management 
education,  was  slightly  modified  as  the  task  in  this  study.  The  case  describes  the  hospital  CEO's  dilemma- 
charged  with  putting  the  hospital's  financial  house  in  1rder,  he  is  faced  with  a  complex  set  of  problems  and 
surrounded  by  advisors  with  conflicting  interests  and  opinions.  Groups  were  -structed  to  recommend  two 
solutions  to  the  hospital  CEO. 

Measures 

NTE  closure  and  NTD  closure  were  the  independent  variables.  Confidence  and  satisfaction  with  the 
decision  were  the  dependent  variables.  Both  were  measured  at  the  .nember-level,  from  member  responses  to 
relevant  items  in  a  post-session  survey.  All  lists  of  brainstormed  ideas  were  collected  for  analysis 

Procedures 

All  groups  used  the  gcnerate-evaluate-select  process  model  during  the  session.  A  facilitator  asked 
group  members  to  read  the  task  instructions  and  the  task  case  study.  Following  a  15-minute  period  for 
reading  the  case  study,  subjects  individually  generated  solution  ideas  and  all  ideas  we:e  recorded  Manual 
groups  used  the  "tablet  method"  to  generate  solutions;  i.e.  jotting  a  solution  on  a  tablet,  returning  the  tablet 
to  a  designated  tablet  area '  on  the  table  and  picking  up  another  tablet  to  record  another  idea.  The  tablet 
method  was  used  to  facilitate  the  exchange  of  ideas  in  an  attempt  to  parallel  the  "Brainwriting"  module 
(VisionqucstTM).  the  electronic  brainstorming  tool  used  by  the  GSS  groups.  They  were  then  asked  to 
work  as  a  group  to  evaluate  all  ideas  and  select  two  prioritized  solutions. 

All  groups  were  given  five  minutes  for  solution  generation  and  20  minutes  for  face-to-face 
discussion  solution  evaluation,  and  prioritization  of  the  tv  o  "best"  ideas.  For  manual  groups,  the  facilitator 
transcribed  all  solutions  from  tablets  onto  a  flipchart.  For  GSS  groups,  solutions  were  captured  via  the 
Brainwriting  module  were  projected  on  a  public  screen.  For  all  groups,  the  facilitator  modified,  edited,  or 
added  to  the  public  solutions  list  as  direct©!  by  the  group.  Each  total  session  took  approximately  90 
minutes.  •  rr  1 
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At  the  end  of  the  session,  each  subject  completed  a  post-sess.on  survey  tot.ncludec If h^Uems 
from  Green  and  Taber's  (1980)  solution  satisfaction  index,  an  instrument  widely  used  in  GSS  research  (e^g 
Zilf DeSa^ctis  &  Billingsley,  1991).  Two  of  the  five  items  (Ql  and  Q2)  measured  members  confidence 
fn  and  sSSSn  with  the  decision  respectively.  Three  items  (Q3.  Q4,  Q5)  tapped  member  commitment 
L^^  rf  responsibility  for,  the  correctness  of  the  decision.  Two  add.Uonal  Uems  focused 
on  session  characteristics  and  member  demographics. 

ReSUlGSS  and  manual  groups  produced  two  datasets  which  were  analyzed  separately  because  of  the  minor 
mrvlifications  made  to  the  procedures  after  completing  the  manual  group  experiment. 

HyZZlis     Hypothesis  1 ,  which  asserted  a  positive  correlation  between  decision  con  idence _and 
decision  Ostein,  wasTupported.  The  Pearson  r  for  both  treatment  groups  was  significant  at  the  .05 
level  (see  Tables  2  and  3). 

Variable       Coeff.    Std.  Error  Std.  Coeff.  _      Tolerance  T  P  (2  tail)  

 "      77^,   77ii  oooo  i-183  0  244 

S0nrHnt      0  02     0    2  0?!?  0.655  4.698  0.000* 

Confidence    0.W2     0.192  ^  ^  ^  ^ 

Q2 


21  °0%l     ^  0,75 


An.filys'*"f  Variance 

SourCe  Sum  of  Squares   DF       MeanSq.  F-ratio 


29.037  3  9.679  9  ,63  0.000* 


Regression               29.03/  J 

Residual                  38.868  38  1.023 

*  significant  at  .05 

DV:  Decision  Satisfaction;  N=42;  Mult.R:  .654;  Squared  Mult.R;  .428;  Adj.  Squared  Mult.R: 
.382;  Standard  Error  of  Estimate:  1 .01 1 

Table  2.  Results  of  Multiple  Regression  on  Confidence  and  Satisfaction  for 
GSS  Groups. 

Variable       Coeff.    Std.  Error  Std.  Coeff.  Tolerance   T  P  (2  tail) 

Cl^nt      2l3ToS»  0.000  2.718  0.010 

Confidence    0.417     0.208  0.381  0.577  2.026  0.J5 

Ql              0.173     0.117  0.263  0.659  1.483  0.148 

Q2             -0.012    0.147  -0.015  0.621  0.080  0.937 

Anf'vsis  of  Variance  0 

Source         Sum  of  Squares  DF       MeanSq.  F-rat.o  J  


3  6.978  5.025  0.006" 


Regression  20.934 
Residual  45.823  33  1.389 

*  significant  at  .05 

DV:  Decision  Satisfaction;  N=39;  Mult.R:  .560;  Squared  Mult.R:  .314;  Adj.  Squared  Mult.R: 
.251;  Standard  Error  of  Estimate:  1.178 

Table  3.  Results  of  Multiple  Rcgresr.kn  on  Confidence  and  Satisfaction  for 
Manual  Groups. 


Correlations  for  NTD  and  NTE  closure  subjects  by  treatment  group  appears  in  Table  4  All 
correlations  are  significant  at  .05.  These  results  suggest  that  ( 1 )  confidence  and  satisfaction  are  significantly 
correlated;  (2)  the  correlation  is  sensitive  to  different  conditions  in  that  it  appears  to  be  stronger  for  need  to 
delay  than  for  need  to  expedite  .  The  last  finding  is  of  particular  interest  considering  that  the  need  to  delay 
closure  treatment  was  operationalized  through  high  (versus  low)  evaluation  apprehension.  The  need  to  delay 
treatment,  by  heightening  evaluation  apprehension  via  the  high  cost  of  judgmental  invalidity  manipulation, 
makes  the  consequences  of  closure  as  an  end  state  more  salient.  Such  consequences  may  be  in  the  positive  ' 
or  negative  direction.  It  follows  that  the  correlation  between  confidence  and  decision  satisfaction  should  be 
even  stronger  under  such  conditions  (relative  to  low  evaluation  apprehension  conditions).  These  results 
provide  strong  support  for  Hypothesis  1 . 


Treatment     n  NTE  Closure      NTD  Closure 


GSS  42         .62*  .87* 

Manual        39         .38*  .64* 


*  significant  at  .05 

Table  4.  Correlations  for  Need  to  Expedite  (NTE)  and  Need  to  Delay 
(NTD)  Closure  Subjects  by  Treatment  Group 


Hypothesis  2.  Hypothesis  2  postulated  that  information  processing  variables  will  not  predict 
decision  satisfaction  directly,  but  rather  through  closure.  For  GSS  groups,  closure  was  a  significant 
predictor  of  decision  satisfaction  (see  Table  5).  On  the  post-session  survey,  Ql  ("We  generated  a  relatively 
complete  set  of  potential  solutions")  and  Q2  ("We  evaluated  alternatives  thoroughly  before  selecting  a 
solution")  were  significant  predictors  of  closure  .  It  is  notable  that  Ql  and  Q2  show  no  association  with 
decision  satisfaction  (p>.05).  Together,  closure,  Ql  and  Q2  explained  38%  of  the  variance  in  decision 
satisfaction. 


Variable 

Coeff. 

Std.  Error 

Std.  Coeff. 

Tolerance 

T 

P(2  tail) 

Constant 

Ql 

Q2 

1.613 

0.236 
0.425 

1.068 
0.084 
0.176 

0.000 
0.387 
0.332 

0.891 
0.891 

1.511 
2.823 
2.419 

0.139 

0.007* 

0.020* 

Analvsis  of  Variance 
Source 

Sum  of  Squares 

DF 

Mean  Sc 

F-ratio 

P 

Regression 
Residual 

14.636 
27.768 

2 
39 

7.318 
0.712 

10.278 

0.000* 

*  significant  at  .05 

DV:  Confidence;  N=42;  Mult.R:  .588;  Squared  Mult.R:  .345;  Adj.  Squared  Mult.R:  .312; 
Standard  Error  of  Estimate:  .844 


Table  5.  Results  of  Multiple  Regression  on  Ql  and  Q2  and  Satisfaction  for 
GSS  Groups. 
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The  results  were  similar  for  manual  groups.  Again,  closure  was  a  significant  predictor  of  decision 
satisfaction  (see  Table  6).  It  is  notable  that  Ql  and  Q2  show  no  association  with  dec.s.on  satisfaction.  For 
manual  groups,  closure  (Ql  and  Q2  together)  explained  25%  of  the  variance  in  decision  satisfaction. 


Variable       Coeff .    Std.  Error 


Std.  Coeff. 


Tolerance 


P  (2  tail) 


Constant  3.006  0.486 
Ql  0.205  0.090 

Q2  0.302  0.110 


O.OOO 
0.341 
0.412 


0.760 
0.760 


6.190  0.000 
2.285  0.029* 
2.758  0.009* 


Analysis  of  Variance 

Source  Sum  of  Squares   DF       Mean  Sq. 


F-ratio 


23.577 
32.098 


2 
34 


11.789 
0.944 


12.487 


0.000* 


Regression 
Residual 

*  significant  at  .05 

DV:  Confidence;  N=39;  Mult.R:  .651;  Squared  Mult.R:  .423;  Adj.  Squared  Mult.R:  .390; 
Standard  Error  of  Estimate:  .972 

Table  6.  Results  of  Multiple  Regression  on  Ql  and  Q2  and  Satisfaction  for 
Manual  Groups. 


In  summary,  the  results  across  GSS  and  manual  groups  were  consistent  with  expectations^ 
information  processing  variables  (Ql  and  Q2)  best  predicted  closure  but  not  decision  satisfaction.  These  two 
variables  and  closure  together  explained  a  significant  portion  of  variance  in  decision  saUsfac  ion.  No 
multicollinearity  problems  were  detected  in  the  analyses  above.  These  results  support  Hypothesis  2. 

An  examination  was  conducted  of  the  link  between  closure,  decision  satisfaction  and  three  variables 
drawn  from  the  Green  &  Taber's  (1980)  solution  satisfaction  index.  These  items  were:  To  what  extent  does 
S [solution  reflect  your  inputs?"  (Q3),  To  what  extent  do  you  feel  committed  to  the  group  solution? 
(Q4)  and  "To  what  extent  do  you  feel  personally  responsible  for  the  correctness  of  the  group  solution? 
05)  For  both  GSS  and  manual  groups,  closure  again  was  the  single  strongest  predictor  of  deciston 
satisfaction.  Q3  and  Q4  showed  moderate  association  with  closure,  but  not  with  the  latter.  These  results 
indicate  that,  from  among  the  post-session  survey  items,  information  processing  variables  (Ql  and  Q2) 
were  better  predictors  of  closure  than  variables  without  an  explicit  information  processing  focus  (Q3.  Q4 
Tnd  Q5)  This  is  fully  consistent  with  our  predictions.  These  results  are  notable  in  that  items  from  a  widely- 
used  scale  show  associations  consistent  with  the  model. 

Analyses  of  Idea  Solutions.  Analyses  of  ideas  generated  by  all  groups  were  conducted.  An  in.t  al 
count  of  individual's  ideas  indicated  29%  more  ideas  were  generated  from  subjects  in  the  GSS  groups  than 
in  the  manual  groups  and  that  only  in  the  GSS  groups  was  there  any  difference  in  the  number  of  ideas 
depending  on  need  for  closure  (see  Table  7),  a  phenomenon  consistent  with  the  literature  (e.g.  Benbasat  & 


Nault,  1990). 

Group 

n 

NTE  Closure 

Manual 

39 

85  (50%) 

GSS 

42 

117(53%) 

TOTAL 

81 

202  (52%) 

NTD  Closure 

85  (50%) 
103  (47%) 
188  (48%) 


TOTAL 

170 

220 
390 


Table  7.    Number  of  Individual  Ideas  and  Percentage  by  Need  for  Expedite 
(NTE)  and  Need  to  Delay  (NTD)  Closure  and  Treatment  Group 

Raters  were  asked  to  categorize  each  brainstormed  idea  solution  on  the  basis  of  whether  it  was  (1 )  a 
single  idea.  (2)  an  "enriched"  idea  [a  single  idea  with  more  breadth  (multiple  steps  or  parts)  or  depth  (detailed 
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explanation  or  clarification)]  or  (3)  multiple  ideas  (more  than  one  idea  contained  in  a  single  thought)  An 

example  for  each  category  taken  from  the  data  appears  below  in  Figure  2 
Single  Idea: 

"Move  all  services  from  the  clinics  to  the  hospital  except  prenatal  care  and  needle 

exchange." 

Enriched  Idea: 

"Close  off-site  clinics  and  move  all  hospital  staff  back  to  hospital,  then  bus  patients  to  the 
hospital.  They  could  still  see  the  staff  they  know/trust." 
Multiple  Ideas: 

"Close  down  all  off-site  clinics.  Set  up  a  shuttle-bus  service  between  the  off-site  clinics  and  the 
hosp.tal.  Use  the  money  saved  for  city-wide  advertising,  new  doctors  &  hospital  improvements." 

Figure  2.  Examples  of  Single,  Enriched,  and  Multiple  Ideas. 

Additional  analyses  were  conducted  on  items  upon  which  there  was  initial  disagreement  until 
consensus  was  reached.  Preliminary  results  appear  in  Tables  8  and  9. 

Treatment  n  Single  Enriched  Multiple 


NTD  Closure      42  64(34%)  64(34%)  60(32%) 

NTE  Closure      39  76  (38%)  79(39%)  47(23%) 


Table  8.  Means  and  Percentages  of  Single,  Enriched,  and  Multiple  Ideas  bv 
Subjects  with  NTD  and  NTE  Closure.  ' 

GrouP  "  Single  Enriched  Multiple 


Manual  39  38  (22%)         68(40%)  64(38%) 

Gss  42  102(46%)  75(34%)  43(20%) 
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Table  9.    Means  and  Percentages  of  Single,  Enriched,  and  Multiple 
Ideas  by  Subjects  in  Manual  and  GSS  Treatment  Groups 

Results  indicate  that  29%  more  total  ideas  were  generated  from  GSS  groups.  GSS  groups  had  more 
single  and  enriched  ideas  than  manual  groups  but  fewer  multiple  ideas  than  manual  groups  It  is  possible 
that  if  the  multiple  ideas  were  teased  apart,  the  results  might  indicate  that  manual  subjects  had  as  many  or 
even  more  ideas  than  GSS  subjects.  Furthermore  NTE  closure  subjects  generated  more  single  and  enriched 
ideas  but  not  multiple  ideas  than  NTD  closure  subjects.  The  need  to  expedite  GSS  group  generated  the 
fewest  number  of  multiple  idea  solutions.  While  it  is  reasonable  that  the  GSS  groups  would  generate  a 
larger  number  of  single  ideas  because  the  technology  provides  limited  space  for  responses  this  theory 
breaks  down  when  comparing  the  number  of  enriched  and  multiple  ideas  which  appear  to  require  equivalent 
space"  for  responding.  Further  analyses  are  needed  to  help  explain  these  results. 

In  summary,  in  this  research  we  advanced  an  operational  definition  of  closure  and  explored  the 
relationship  between  confidence  (as  a  cognitive  belief  preceding  closure)  and  satisfaction  in  group  decision 
making  using  GSS  and  manual  methods  in  an  experimental  setting.  A  theoretic,  cognitive-motivational 
framework  was  used  to  conceptualize  the  constructs  of  interest  in  a  closure  model  of  decision  satisfaction 
The  results,  which  were  consistent  with  expectations  based  on  the  model,  suggest  that  (1)  confidence  and 
decision  satisfaction  are  significantly  correlated,  and  (2)  information  processing  variables  are  significant 
predictors  of  confidence,  but  not  of  decision  satisfaction.  Together,  confidence  and  two  information 
processing  variables  explained  a  signficant  proportion  of  the  variance  in  decision  satisfaction. 

Discussion 

The  Cognitive-Motivational  Model  of  Group  Member  Decision  Satisfaction  characterizes  the 
decision-maker  as  an  active  evaluator  of  the  adequacy  of  the  "epistemic"  process.  Need  for  closure  influences 
adequacy  assessment.  Information  processing  deemed  extensive  in  need  to  expedite  conditions  may  be 
deemed  inadequate  in  need  to  delay  closure  conditions,  suggesting  that  the  uncertainty  cutoff  may  be  set 
relatively  low  in  need  to  expedite  conditions  and  relatively  high  in  need  to  delay  closure  conditions. 
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This  model  suggests  a  number  of  relevant  areas  for  future  research  on  satisfaction  and  the  use  of 
technology  support  sysfems  for  enhancing  satisfaction  with  group  work  and  further  development  of  our 
model  One  promising  area  is  satisfaction  as  a  criterion  for  assessing  other  mot. vation  variables^  For 
example  Ro«o  0       and  Arnone  &  Small  (1995)  posit  a  relationship  between  satisfaction  and  cunos.ty. 
Sdual  differences  (e.g.trait  curiosity,  impulsivity ,  tolerance  for  ambiguity)  may  mfluence  need  for 
closu  e  and  therefore,  should  be  explored.  Furthermore,  this  study  explored  how  potential  negative 
^jSXSta«««)  affected'extent  of  information  processing;  but  would  there  be  similar  .esults 
with  potential  positive  consequences  (reward)? 

Research  is  needed  to  investigate  group  member  attitudes  (Hooper  et  al .  1 994). 

Informal  discussions  with  some  subjects  indicated  annoyance  with  attitudes  oi  some  team  members  (e.g. 
doZalg  lack  of  empathy).  The  impact  of  group  dynamics  and  interactions  and  of  ad  hoc  groups  vs. 
estTbliS  teams  on  confidence  and  satisfaction  are  potentially  important  areas  for  additional  study. 

Computers  also  facilitate  feedback  and  group  interaction.  Although  the  current  model  does  not 
incorporate  a  feedback  component,  adding  appropriate  feedback  loops  to  the  model  may  increase  the 
likelihood  of  confidence  and  satisfaction  in  group  work.  This  is  another  area  for  future  work, 
likelihood  ot  cont  satisfaction,  it  might  be  interesting  to  explore  the 

relationship  between  closure  and  process  satisfaction.  If  need  for  structure  is  functionally  equivalent  to 
need  foi  closure  (Mayseless  &  Kruglanski,  1987),  then  interventions  emphasizing  process  structure  may 

promote  g^«JJ^     (0  organize  and  quickly  retrieve  data,  computers  are  believed  to  hold  great 
promise  for  creating  environments  for  developing  and  enhancing  problem-solving  and  decision-making 
5  Duffield  1994;  Flake  et  al.,  1985;  Thornburg,  1986).  The  study  of  satisfaction  may  also  have  direct 
£&£^  group  work  and  the  development  of  computer-based  systems  that  support  group  problem- 
solving  and  decision-making  in  learning  contexts. 

solving  ana  |         ^  ^  ^  ^  of  declsIon.making ;  skllls  t0  a 

computeSulated  environment  intended  to  help  young  students  working  in  pairs  to  reduce  impulsivity  in 
dSn  maSng  situations  and  make  more  thoughtful  decisions.  They  found  instead  that  in  the  computer- 
SSLfSSIent  there  was  a  significant  increase  in  impuslivity;  i.e.  decision  making  by  one  partner 

"°  ^S^^^'^  have  occurred  due  to  their  pretest  session  experience  with 
the  same  ''which  could  have  provided  them  with  a  greater  degree  of  confidence  to  make  decisions  without 
Sf^ShsS^  60).  They  attribute  much  of  the  impuslivity  to  the  effects  of  computer  games  which 

dedsioi  in  order  to  avoid  the  penalties  imposed.  They  suggest  a  "braking device  to 
slow  down  the  decision-making  process  and  the  addition  of  explicit  prompts  to  cause  s  udents  to  be  mere 
SectrThe  auton  suggest  that  the  decision-making  process  is  more  than  acognitrve  activity  but  that  ,t 
ako  incoroorates  other  aspects  of  an  individual,  including  affect. 

also  '"corporate^     ^  future  research  mat  expiores  the  use  of  technology  in  group 

learnine  situations  as  well  as  performance  and  motivational  outcomes  in  group  work.  Research  that 
exXesS  *e  cognitive  and  affective  aspects  of  higher  order  thinking  skills  such  as  problem-solving  and 
dec  ^on-making  will  contribute  to  the  development  of  computer-based  systems  that  enhance  those  skills. 

Computer  systems  are  needed  that  go  beyond  merely  displaying  generated  ideas  to  visually 
mappingSdC  so  that  users  can  sec  interrelationships  among  ^^f.^^^^  " 
depth  relevance,  and  usefulness  of  individual  solutions.  In  addition.  Steeb  and  Johnston  (198 1)  suggest 
that  visuTaid  organizers  may  help  decision-makers  deal  with  information  overload  more  effectively, 
that  v>sua^o0rg.on  of      *cations  of  GSS  technology  in  education  may  have  particular  implications  for 
instruction  using  case  studies  (e.g.  management  education).  Case  study  analysis  lends  itself  we 1  to 
nvestigation  of  group  member  satisfaction  since  it  often  involves  group  analysis  and  judgment 
;S  cL  study  method  involves  analyzing  and  processing  compkx  infor ^a«on  generaung 
plSle  alternative  problem  solutions,  and  making  decisions  (Hudspeth  &  Kn.rk,  1989;  Hammond,  1980. 
Sulkin,  1966)  ^  ^  need  tQ  teach  informa,ion  ski„s  For 

example  Sieber  &  Solomon  (1970)  and  Lanzetta,  (e.g.  1963)  assert  that  teaching  students  skills  such  « 
cue  disc  imination,  hypothesis  generation  and  hypothesis  evaluation  and  information  skills  such  as  f.ltenng 
SSL  relevantinformation  will  lead  to  greater  information  search  behav.or  and  decrease 
uncertainty  ^  ^  ^  ^  ^  rf        by  t  and  usc  of 

GSS  or  manual  methods.  Future  research  will  include  analysis  of  the  quality  of  ideas  generated. 
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About  ten  years  ago  Nolan  Bushnell,  the  founder  of  Atari  and  microelectronics  pioneer,  was  the 
keynote  speaker  for  AECT.  One  striking  picture  that  Bushnell  portrayed  of  educational  technology  in  the 
future  is  the  ability  through  virtual  reality  to  come  close  to  first  hand  experience  of  any  event  or  place. 
Instead  of  reading  about  Italy,  a  student  will  be  able  to  walk  down  the  streets  of  Italy  where  (s)he  wishes  to 
go,  smelling  the  smells,  feeling  the  sunshine.  Instead  of  reading,  hearing  about,  or  passively  watching  a 
reenactment  of  the  "Boston  Tea  Party"  students  could  actually  "participate"  in  it,  hacking  open  barrels  of  tea 
and  tossing  them  into  the  Boston  Harbor.  Being  true  fans  of  the  "Star  Trek  Next  Generation"  holodeck  we 
have  been  enchanted  by  the  educational  future  that  Bushel  presented.  At  the  same  time,  as  educational 
technologists,  instructional  designers,  learners,  and  tax-payers,  we  were  immediately  both  struck  by  the 
effort  involved  in  such  virtual  experiences  and  by  the  uncertainty  of  when  such  experiences  would  be  so 
desirable  that  the  costs  would  be  worthwhile.  We  have  also  conjectured  about  if  or  when  such  experiences 
might  actually  be  deleterious  to  learning.  Although  these  issues  do  not  "map"  directly  upon  the  issues  of 
situated  cognition,  they  are  related  to  some  of  the  recurring  issues  addressed  by  instructional  designers  and 
instructional  theorists  regarding  the  role  and  nature  of  experience  in  learning. 

Recent  developments  in  microelectronic  technology  have  made  substantial  changes  on  how  experience 
and  information  are  perceived.  These  changes  have  also  contributed  to  a  major  shift  in  how  instruction  is 
perceived.  Instructional  technologists  must  keep  abreast  of  not  only  the  product  technology  developments 
and  their  impact  on  popular  culture  but  also,  and  perhaps  most  importantly,  the  implications  and 
repercussions  for  instructional  technology.  It  is  easy  to  become  swept,  up  in  the  flow  of  the  more  widely- 
held  popular  interpretations  of  "technology,"  loosing  both  traditional  strengths  and  intellectual  leadership  so 
desperately  needed  in  times  of  change.  In  this  paper  we  will  consider  the  relative  cognitive  requirements  and 
benefits  of  learning  from  non-contrived  experience  and  information  presentation,  and  contrast  these 
advantages  and  costs  with  contrived  experience  and  information-reduction  that  are  features  of  instruction. 

In  our  reflecting  on  these  issues  we  have  constructed  one  perspective  of  how  experience  and  information 
relate  to  each  other  and  to  instruction.  We  later  in  this  paper  will  discuss  these  terms  but  here  we  will 
overview  one  way  of  viewing  these  relationships  that  may  be  productive  for  instructional  technologists. 
The  dimension  represented  on  the  continuum  is  abstraction  of  experience.  In  "everyday  experience"  the  only 
abstraction  that  occurs,  is  that  which  the  individual  performs  on  the  raw  stimulus  data.  On  the  other  end, 
information  is  data  that  has  been  reduced  and  organized  by  the  author  in  order  to  represent  his  or  her 
personally  relevant  interpretation  of  multiple  events,  objects,  and  experiences. 


Figure  1.  Experience  to  information  continuum. 
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EXPRecemdevelopments,  and  much-discussed  pending  ones,  in  computer  technology  are  a 
person  a  ^ticipa'e  in  the  way  of  feasible  experiences.  Increasing  precIS1on  and^  m  of  a  aHaMe 

SS II  computers:  Thi/technology  can  no .longer  - ^considered 
"Artificial  experience  "  whether  represented  by  aircraft  simulators,  chess  players  city-bu- .  1,_ 
Sg  SSfoSTSSt  courses  in  the  word,  or  engaging  in  a  worldwide  search  for  a  ma  criminal 

^  tul^li^  25  -hnologys  impact  on  what  we  may 

experience  V^rtua  reality  maximizes  sensory  involvement  in  increasingly  realistic  and  <^P^f 

8       coan ove o^rSon  of  forming  concepts  is  a  means  of  proving  us  from  expenencmg  everyttag 

t^nS  musical  visual  -  allows  condensation  and  distillation  of  direct  experience.  Most  of  the 
m     ,w SiTS SCua«  systems  can  be  performed  without  them,  but  at  impossible  expense  m  time 

ISanon  o^utafs  exp.*noes.  Dewey's  work  on  the  role  of  expenence ■»  kaimng  (19.9,  1938) 
^Sbi=nS= 

S  SSond  tStogy  provides  us  with  a  changing  conception  of  the  role  of  expenence.  A  long- 

How  we  come  to  know  is  seen  to  be  in  our  interactions  with  the  environment,  not  a  product  ot  those 
"^wfconcur^h^ririportant  contribution  to  the  quality  of  learning  that  situated  cognition  auntie 

SSffittS«»  th,t  situated  cognition  and  other  constructivist  tools  represent  true  progress  ,n 


this  direction. 
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*m™^fuU]Z  r  faC!  iSe3  IaCk,°f  SUff'Cieni  W°rk  °n  when'  and  under  what  circumstances  these  tools  or 
approaches  should  be  used.  Some  alternative  conceptualizations  could  be  explored- 

1 )  Approach  all  .nstruction  from  either  extreme,  only  information,  always  direct,  everyday  experience 
We  reject  this  alternative  out  of  consideration  for  the  requirements  imposed  by  certain  interactions  of 
learners,  learning  tasks,  and  often  other  factors  from  instructional  needs,  settings  and  the  like  Other 
considerations  may  also  tip  the  choice,  including  those  for  need  for  efficiency  of  learning  that  may  override 
needs  for  mean.ngfulness  and  interest.  Medical  schools  may  choose  not  to  break  each  bone  in  the  bodies  of 
s  udents  who  are  specializing  in  orthopedic  surgery.  Or,  as  Winn  has  noted:  "I  am  not  convinced  yet  that 
all  knowledge  can  be  constructed  by  students.  The  student  must  have  some  knowledge  from  which  to  start 
instruction,  and  that  knowledge  must  be  explicitly  taught."  Winn  (1992) 

2)  Never  acknowledge  one  of  the  extremes  as  a  means  to  learning.  We  reject  this  alternative  based  on 
consideration  of  the  unquestionable  value  of  direct  experience  and  raw  information  to  learning  However 
we  would  not  term  these  extremes  "instruction." 

3)  Sometimes  use  experience-rich  or  information-rich  strategies,  depending  on  the  situation    This  is 
the  position  we  wish  to  work  from,  leaving  the  question,  what  would  guide  us  in  deciding  when  and  under 
what  circumstances  will  working  at  the  extremes  be  useful?  The  task,  it  seems  to  us,  is  to  learn  more 
about  what  interacts  with  situations  and  instructional  strategies. 

Information 

Popularly  available  information  in  recent  years  has  changed  from  static  to  dynamic,  from  carefully 
gatekept  to  open,  from  delayed  to  immediate,  and  from  structured  to  free-form.  We  see  this  information 
resident  in  hypermedia,  databases,  computer  networks  and  bulletin  boards,  and  in  r  formation  storage  media 
such  as  CD  ROM  technology.  This  information  can  be  timely,  provide  contrasting  perspectives  and 
provide  extreme  depth  and  detail  in  coverage.  To  those  having  strong  information  needs,  never  has 
information  been  so  rapidly  available. 

Exploration  in  information-rich,  hypermedia  environments  has  been  seen  by  some  as  a  solution  to 
educational  problems.  However,  provision  of  access  to  information,  regardless  of  how  deep  or  elaborate  is 
not  in  and  of  itself  sufficient  for  instruction.  All  of  the  issues  that  the  constructivist  movement  has  so  ' 
forcefully  put  are  germane:  meaning  does  not  reside  in  the  message,  ideas  are  not  "transferred,"  individuals 
must  organize  or  "construct"  information  in  ways  that  are  personally  meaningful.  This  task  can  be 
overwhelming  when  working  from  an  information-only  base,  or  as  Perkins  described  it  having  a  "hieh 
price  in  ihe  cognitive  economy"  (1992). 

Some  research  seems  to  support  the  suggestion  that  pure  information  may  not  be  sufficient  to  support 
learning.  For  example,  Jacobson,  Kolar,  Maouri,  Mishra  and  Spiro  (1994)  investigated  learning  in 
hypertext  environments  and  found  that  learners  in  a  no-guidance  condition  were  overwhelmed  and  not  as 
successful  as  learners  provided  with  guidance.  "The  mere  availability  of  nonlinear  hypertext  links  may  not 
be  sufficient  to  affect  substantive  learning  outcomes."  (p.  17) 

Some  limitations  inherent  to  information-based  learning  systems  include  the  expectations  of  learners' 
abilities  to  select,  evaluate,  sequence,  and  structure  this  information  anticipates  new  levels  of  text 
processing  and  other  learner  strategies  that  may  not  be  presently  available  in  many  learners'  repertoire 
Other  characteristics  possessed  by  novices  in  a  domain  severely  limit  the  usefulness  of  unguided  exploration 
in  information  bases. 

Instruction 

The  technology  of  printing  reduced  demands  on  learners  from  that  of  direct  experience  or  the 
vicissitudes  of  oral  tradition,  and  the  technology  of  instruction/teaching  has  offered  a  guiding  hand  through 
the  world  of  information.  The  term,  "instruction,"  never  well-understood,  seems  to  be  taking  a  new 
meaning,  one  that  is  far  from  its  rich  meaning  in  the  context  of  instructional  technology.      A  frequently- 
observed  trend  is  to  consider  instruction  as  referring  to  only  a  specific  sort  of  strategies,  ones  of  a  didactic  or 
reception-learning  sort.  By  some,  instruction  is  perceived  as  something  "done"  to  learners,  rather  than 
something  that  is  "participated  in"  by  learners.  We  consider  instruction  to  include  a  range  of  strategies  that 
facilitate  experiences  that  are  expected  to  lead  to  learning. 

Instruction  may  be  seen  to  reside  between  two  poles:  experience  and  information,  and  should  involve  a 
process  that  involves  both  sources  of  learning,  as  the  situation  requires.  On  the  continuum  presented 
earlier,  we  place  instruction  in  the  range  between  everyday  experience  and  transmission  of  information. 
Indeed,  we  describe  instruction  in  this  arena,  to  be  the  facilitation  of  contrived  experiences.  These  contrived 
experiences  can  mnge  from  "being  there"  as  in  participating  in  the  Boston  Tea  Party  to  selecting  and  reading 
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excerpts  of  diaries  of  British  and  colonial  participants  and  observers  of  the  Boston  Tea  Party,  to  reading 
historical  analyses  of  this  event  and  evaluating  or  reacting  to  them. 

Instruction  that  is  all  experience  is  at  risk  of  being  vapid  and  impractical,  or  too  limited  in  scope  or 
example-provision  to  allow  transfer.  Instruction  that  is  all  information  may  be  irrelevant  and  meaningless. 
Choices  between  the  two  sources  as  well  as  other  essential  ones  in  fostering  learning  are  implicit  in 
instructional  design.  The  richness  of  experience  and  information  that  computer-based  learning  environments 
can  now  provide  is  seductive  in  appearing  sufficient  to  learning,  in  much  the  same  way  that  motion  picture 
films  were  seductive  to  educators  of  the  1920's  and  30's.  Fascination  with  the  power  of  a  technology 
should  not  distract  us  from  learning  to  harness  the  technology  for  learning  rather  than  worship  it.  Critical 
in  these  instructional  decisions  are  evaluating  the  nature  of  the  learners'  prior  knowledge,  motivations, 
beliefs  as  well  as  the  nature  of  the  potential  learning  outcomes. 

A  question  illustrative  of  our  concern,  is  what  forms  of  learning  guidance  are  needed  in  exploratory 
learning  situations.  We  are  currently  engaged  in  empirical  studies  on  this  topic,  in  which  varying  forms  of 
support  are  employed  in  exploratory,  computer  based  learning  environments.  Our  preliminary  results  are 
pointing  to  a  particular  needs  for  support  for  novice  learners,  different  from  the  needs  of  learners  with  prior 

knowledge  or  skill.  .  ,.  ... 

In  an  article  on  conditions  that  foster  student  engagement  in  productive  autonomous  learning  activities, 
Thomas  and  Rower  (1993)  provide  a  set  of  course-specific  facilitators  including  the  explicit  articulation  ot 
demand  conditions  and  support  practices,  the  provision  of  authentic  self-directed  learning  responsibilities  and 
tasks  and  opportunities  to  assess  the  effectiveness  of  existing  strategies  and  to  learn  new  ones. 

Often  the  provision  of  the  paired  situations,  as  exemplified  in  the  Jasper  series  (such  as  Journey  Down 
Cedar  Creek),  while  rich  and  engaging,  fails  to  provide  application  of  skills  in  more  than  one  context, 
deceasing  the  odds  of  transfer.  We  would  not  avoid  the  Journey  Down  Cedar  Creek  (though  a  question 
exists  as  to  how  "authentic"  a  situation  a  boat  trip  might  be  to  many  learners,  such  as  those  from  inner  city 
environments),  but  would  question  its  adequacy,  hoping  to  extend  it  with  practice  using  multiple  examples. 
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The  overall  goal  of  this  study  is  to  show  how  some  concepts  and  techniques  from  disciplines  outside 
InrtSc^K  Dewtopnit  (ISD)  have  the  potential  to  extend  and  enhance  the  tradu.onal  view  of 
SD  p  actice  when  they  are  employed  very  early  in  the  ISD  process.  The  concepts  and  techniques  we 
employS wereuser  Entered  design  and  usability,  and  the  context  of  our  study  was  an  mstruct.onal 
development  project  in  the  earliest  stages  of  design. 

Wh3t  ^^T^^^^^so^  that  places  the  eventual  user  of  a  product  at  the 
heart  of  the  des  en  process  for  that  product.  Usability  is  a  key  concept  within  the  philosophy,  and  refers  to 
S^offiESlLly  considered  to  determine  the  ease-of-use  a  product  w,  enjoy^earnab,  ty, 
efficiency  memorability,  error  prevention  and  recovery,  and  user  satisfaction  (Nielsen,  1993  .  The  terms 
f  sometime'sed  interchangeably,  and  both  are  used  frequently  to 

methods  and  orocesses  that  facilitate  the  creation  of  usable  computer  products  (S  rong,  995)^  The  metnoas 
■  ™  P^ssesP™ed  with  user-centered  design  are  derived  from  multiple  disciplines  mcludtng,  among 
£s  human  factors,  ergonomics,  psychology,  ethnographic  studies,  and  document  des.gn. 

One  of  the  fundamental  principles  of  user-centered  design  is  that  users  must  be  involved  in  design 
decisions  for  the  product"  they  will  use.  From  this  principle  is  derived  an  iterat.ve  approach  to  de  .gn, 
•      th?i„™,t  ^^f  uLrs  forces  designers  to  reconsider  their  own  decisions  and  assumptions  during  the 
nC£ "i^^SSSf  *J  terative  approach  demands  use  of  rapid  prototyping  techniques  to 
^S^-SSrifcnough  in  £  process  that  they  may  be  altered,  sometimes  repeatedly, 
before  they  are  delivered  in  final  form. 

the  development  of  an  instructional  product. 

AoDivine  user-centered  design  to  the  concept  of  an  interactive  whiteboard 

APPTh^ Vi  rt Z  textbook  project'stiil  in  early  design  stages  is  defined  ^^^J^Smn,  of 

r^S^SSto  As  it  s  conceptualized,  students  would  transmit  text  and  pictures,  among  o^er 
fffe^^Svirial  textbooks  onto  the  whiteboard  and  share  these  items  wdh  other  members  of 
their  group. 

There  have  been  some  references  to  interactive  whiteboard  technology  in  corporate  settings  (e.g 
Wei£  W ?)  but  w hTve  found  no  published  studies  on  incorporating  an  interactive  whiteboard  ,n 
SSnnrf  ettines'  hence  the  interactive  whiteboard  is  a  novel  iesign  problem  for  this  team. 
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Our  major  goal  for  the  interactive  whiteboard  study  was  to  understand  hnw 
=^  inactive  whiteboard 

interactive  whiteboard  s  Z  use  of  th     1!    a  S  mterfaCe-  We  chose  t0  simulate  the 

established  intc^iSZuZlX^^^^  7^  w°f  ^  ■»  to  conform  to  an 
down  pictures  and  text  from  a  regular  black^rTin     Whlteboard' we  used  volunteers  who  put  up  and  took 
the  study  were  told  to  dtaS  S^u^^JSSS?  TTtX  StUdents  Pupating  in 

minima,  instruction  on  formula*  panose  ^  °f  the  volunteer-  »«■  given  only 

»n,u,attd  „hiKb„,«,.  Su8„  and  Bo,i„8  (i„  J^^^^^E"  ««■ "* 
Results  of  usability  sessions 

from^rusSns ^^^^^J^X^^^  ^  h0m  the  sessions  ^  ™** 
.social  issues.  V^^^^^S^^^S0^  T  gr°Uped  int°  ^Regies/tactics  and 

usability  session  videotapes  and  reco  SS  ^^T^*^1*"  a,°ng  Whh  Several  °f  to 
proposing  new  and  revised  feam  ^  S  r  J     I  designer's  tenons  fell  into  four  categories,  1) 

uestionin the  design  o   he  usa  ,'ity  e'Tons  3^3^         « ^  *e  3 
sessions  usaomty  sessions,  and  4)  posing  questions  to  be  answered  in  future  usability 

Findings  from  the  usability  sessions 

Slrategies/tactics 

simultaneous  moves  (those  moves  that  were  voiced  by  more  than  one  subject)  ]' 

'Command  language:  Each  student  group  use  a  similar  command  language  pattern  in 
communing  to  the  shared  space.  In  the  beginning  of  the  task,  studenJconCc,  d  their 

S3 £  ^  fromt SUbj£Ct"  'ST  ^  7*  ^  *^ ln  1 950")  S^ud"  s  g  ad  ally 
Be?v  in  19?S  t     T™^  f°rmat  and  referred  to  the  materia>  by  the  subject  (e.g.,  '  Chuck 
Berry  ,n  1950  ).  When  student  groups  were  involved  with  task  and  not  cognizant  of 

^Zed'pacV^S,:^*  thCy  rdied  °"  Inicate  with 

SS^T^r^SJ^  °f  J3rg0n  mC,Uded  SUCh  St3tementS  aS:        -  ^e 

•  Students1  tmgaage.  During  group  discussions,  the  volunteer  simulating  the  interactive 
whiteboard  could  not  always  tell  whether  students  wanted  to  put  nn  item  o  the board  One  student 
would  say  that  they  should  put  an  item  up,  but  another  stud.m  would  dTsagree  an    he  lan^ute 
they  employed  m.ght  or  might  no,  be  construed  as  direct  commands  to  he IterfacV        *  * 
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.  Shared  space  ,naintenance:  Since  the  student  groups  only  had  a  limited  amount  of  space  (a 
maximum  of  21  items),  the  groups  eventually  confronted  the  issuing  of  cleaning  up  or 
mLnZTng  th   S  spacf .  The  two  groups  employed  two  major  strategies  .n  dealing  with  this 
Sue  Some™  dent  groups  issued  global  commands;  for  example,  one  group  asked  to  take  all  of 
SJSlSlI  Shared  space  (e.g.,  "Take  all  of  the  written  statements  down").  Another 
major  strategy  was  to  designate  a  certain  portion  of  the  shared  space  to  review  items. 

.  hem  maintenance:  Each  student  group  employed  common  tactics  in  maintaining  their  work 
pice  The  e  was  an  apparent  need  to  sort  through  their  items  on  their  desks.  Students  would  »ft 

Kerns  £  figure  out  which  items  to  include  on  the  timeline;  they  would  put  used  items  ,n  a 
enarate  D  le  TOs  finding  and  the  previous  one  indicate  a  need  for  a  workspace  to  v,ew  several 

f^TO^Xofkspacc  could  be  located  on  student's  individual  virtual  textbook  or  on  the 

common  shared  space. 

.  Shared  space  navigation:  Before  students  would  put  up  a  item,  they  checked  the  whiteboard. 
T^ere  was    consent  interplay  of  students  checking  the  board,  their  respective  piles  of  pictures  and 
^cSn^SSISin,  and  then  deciding  which  item  needed  to  be  on  the  board^  Student 
also  needed  to  figure  out  a  specific  location  to  put  their  items  on  the  shared  space.  Students  either 
SrS  to  a  specific  year  on  the  timeline  or  referred  to  a  existing  item  on  the  whiteboard  near 
which  to  place  pictures  or  text. 


Social  aspects 

.  Common  ownership-  A.  major  question  prior  to  this  study  was  whether  students  had  common 
owneThio  of  the  materials  that  were  held  in  shared  space.  That  is,  if  one  student  put  up  his  or  her 
Stall  harXSi  wouW  another  student  freely  move  or  take  down  that  student's  p.cture? 
ShelmrnXSchstudent  group  treated  the  shared  space  material  as  common  property 
StfmSSl  «hor  students'  materials  to  other  locations  in  the  shared  space  and  deleted  those 
materia  Is  off  the  board.  When  there  was  a  disagreement  on  the  transfer  or  removal  of  a  certain 
Uem,  it  wa"setUed  by  the  dominant  group  members  in  that  student  group,  not  the  ong.nal 
"owner"  of  the  picture  or  text. 

.  Turn-takinr  We  expected  to  see  students  establish  a  set  of  rules  on  how  to  take  turns  in 
comlSng  with  L  shared  space.  It  appears  that  there  were  usually  one  totwo  dominant 
Srwho  acted  as  "gatekeepers"  or  spokespersons  for  each  student  group.  These  students 
wouS  dkeSoSstudLs  moves.  They  would  either  approve  or  veto  the  group's  decisions  This 
ulnl  of  po-e  ^originated  from  students'  need  to  confirm  decisions  among  group  members  before 
puSg  item!  on  the  board.  These  decisions  were  mediated  through  these  dominant  student(s). 

.  Strateev  discussions-  After  the  initial  period  of  putting  items  on  the  board  students  usually 
£SS  on  how  to  complete  the  timeline.  These  discussions  focused  on  particular 
pieces  (e  g  ,  "who  has  Jimi  Hendrix?")  and  where  to  place  items  (e.g.,  "where  should  we  put  Bob 
Dylan?"). 

.  Limited  collaborative  skills?  The  student  groups  understood  their  task  to  be  a  cooperative  one. 
since  thev  were told  that  they  each  had  part  of  the  materials  required  to  complete  the  timeline,  and 
n To  J of  them  ha Everything  required  However,  none  of  the  groups  was  observed  to  organize 
^W^pSSto.  for  the  work,  choose  a  leader  (although  dominant  members  emerged  in 
e7ch  erouo-  settled  on  or  even  discuss  a  process  for  their  work,  or  carry  out  any  coherent  effort  to 
pLn  the S  Various  groups  were  observed  to  spend  time  ineffectively,  marginalize  one  or  more 
membSs  who  were  latfr  discovered  to  have  information  needed  by  the  group,  and  backtrack 
through  actions  due  to  misunderstandings. 

.  Need  to  consult  at  close  range?:  Even  though  there  was  a  restriction  on  sharing  hems  among 
themselves  (in  order  to  maximize  observed  interactions  with  the  whiteboard  and to .simulate  the 
probability  that  virtual  textbook  screens  might  not  be  large  enough  to  use  for  this  kind  of 
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consultation)  and  groups  were  reminded  of  the  restriction,  students  consistently  consulted  with  each 
other  by  holding  up  and  passing  around  pictures  and  text  items. 


Results  from  discussions  with  Virtual  Textbook  designers 

The  virtual  textbook  designers  had  four  major  types  of  reactions  to  our  presentation  of  these 
findings  and  their  viewing  of  videotaped  sessions;  1)  proposing  new  and  revised  features,  2)  reconsidering 
assumptions  about  students  using  the  whiteboard,  3)  questioning  the  design  of  the  usability  sessions,  and  4) 
posing  questions  to  be  answered  in  future  usability  sessions.  We  believe  these  reactions  gives  insight  into 
the  potential  impact  of  the  usability  sessions  on  the  process  of  designing  instructional  products  and  the 
possible  impact  of  user-centered  design  on  the  instructional  systems  design  (ISD)  process. 


•  Proposing  new  and  revised  features  for  the  whiteboard:  The  designers  had  envisioned  the 
functionality  of  the  interactive  whiteboard  within  the  virtual  textbook  environment  to  the  extent  of 
representing  some  of  its  major  features  in  a  "conceptual  video"  mock-up,  but  once  they  were 
presented  with  findings  of  the  usability  sessions  and  the  videotapes,  they  began  to  speculate  on 
both  new  and  dramatically  revised  features  the  whiteboard  might  have.  Among  other  things,  they 
discussed  the  development  of  a  new  command  set  within  whiteboard  interface,  and  designing  a 
horizontal  whiteboard  instead  of  a  vertical  one. 


•  Reconsidering  assumptions  about  students  using  the  whiteboard:  The  designers  showed 
considerable  interest  in  the  findings  concerning  strategies/tactics  and  social  aspects  described  in  the 
preceding  section  and  spent  time  discussing  how  to  redesign  and  reconceptualize  the  interactive 
whiteboard  in  response  to  this  more  detailed  view  of  the  students  who  would  use  the  technology. 
Should  the  whiteboard  interface  try  to  compensate  for  limited  collaborative  skills?  If  so,  how? 
Should  the  whiteboard  interface  encourage  or  discourage  the  "gatekeeper"  function  observed  in 
every  group? 


•  Questioning  sessions'  design:  The  designers  wondered  whether  students  were  comfortable  with 
addressing  the  "simulated"  whiteboard  and  suggested  that  future  usability  sessions  should  focus  on 
students  groups  working  without  an  interactive  whiteboard  to  answer  some  of  their  new  questions 
about  how  students  work  together  and  to  limit  the  admittedly  confounding  effect  of  the  simulated 
technology.  The  designers  were  enthusiastic  about  the  possibility  of  conducting  further  usability 
sessions,  in  spite  of  their  questions  about  the  design  of  this  one. 


•  Posing  questions  to  be  answered  in  future  usability  sessions:  The  designers  posed  a  variety  of 
questions  they  wanted  answered  through  future  studies:  do  students  need  to  touch  their  documents 
rather  than  indirectly  manipulating  them?  how  should  students  cue  their  commands  with  the 
interface?  what  type  of  order  needs  to  be  imposed  with  a  common  work  space?  will  the  same 
social  dynamics  occur  among  students  if  the  usability  sessions  were  altered?  what  impact  is  the 
whiteboard  technology  having  with  the  social  dynamics?  Though  there  were  no  conclusions 
among  designers  about  the  design  of  the  Virtual  Textbook  interactive  whiteboard  interface,  there 
was  tremendous  amount  of  creativity  exhibited  by  the  design  team  during  the  discussions  of  these 
usability  sessions. 
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Early  usability  testing  model 

The  model  represented  in  this  paper  describes  how  we  conducted  usability  tests  in  relation  to  the  design 
and  development  work  ongoing  in  the  Virtual  Textbook  project.  Although  our  study  used  two  separate 
teams  to  conduct  tests  and  carry  out  ongoing  development,  the  model  could  as  easily  describe  parallel 
activities  by  members  of  a  single  team.  The  model  cycles  back  into  itself,  demonstrating  the  difference 
between  usability  studies  and  experimental  research;  the  focus  of  usability  studies  is  contained  within  a 
project  and  the  results  of  the  studies  guide  further  design  efforts.  There  is  no  expectation  that  findings  from 
these  studies  will  generalize  outside  the  project  for  which  they  are  conducted,  although  they  may  be  of  great 
interest  to  designers  working  on  similar  projects. 
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Some  anticipated  criticism  of  the  early  usability  testing  model  is  that  this  testing  is  simply  formative 
evaluation.  Tessmer  (1993)  points  that  the  rapid  prototyping  technique  is  similar  to  "repeated  formative 
evaluations."  Although  the  evaluation  techniques  are  similar  in  usability  testing  and  traditional  formative 
evaluation,  the  key  difference  between  the  two  is  how  early  in  the  process  and  how  often  a  product  is 
evaluated.  In  describing  the  rapid  prototyping  techniques  of  his  ID2  theory,  Merrill  and  his  associates 
comment  that  the  instructional  systems  development  (ISD)  approach  assumes  that  there  will  be  no  actual 
product  until  the  later  stages  of  the  process  ('ones,  et  at,  1992).  The  early  usability  testing  model  provides 
designers  the  opportunity  to  assess  and  evaluate  their  concepts  and  assumptions  during  the  earliest  stages  of 
a  design.  The  model  also  presumes  that  the  evaluation  process  will  be  carried  out  several  times  during  the 
development  process,  giving  designers  the  opportunity  to  adjust  designs  in  situations  where  they  have  little 
prior  experience  to  gauge  the  probable  success  of  a  product. 

Implications  for  the  ISD  process 

What  impact  does  this  model  have  for  the  ISD  process?  In  our  experience  with  this  study,  early 
usability  testing  may  impact  the  ISD  process  in  four  ways;  1)  condense  ISD  process;  2)  refine  prescriptive 
methods;  3)  encourage  a  cyclical  process;  and  4)  concentrate  attention  on  the  human  elements  of  the 
process. 
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•  Condenses  ISD  process:  The  results  of  user-centered  design  evaluations  provide  insight  for  the 
next  analysis  phase,  and  can  give  rise  early  in  the  process  to  questions  that  might  not  otherwise  be 
asked  until  a  product  is  almost  ready  to  be  implemented.  Catching  and  considering  a  wider  range 
of  issues  for  design  at  the  beginning  of  a  project  may  reduce  overall  development  time,  especially 
in.  cases  of  novel  or  problematic  technology. 

•  Refines  prescriptive  methods:  Early  usability  testing  also  gives  instructional  designers  the 
opportunity  to  test  the  prescriptive  methods  they  are  encouraged  to  develop  by  Reigeluth  (1983). 
Through  each  redesign  phase  of  the  early  usability  testing  model,  designers  can  refine  these 
method'  with  the  goal  of  having  the  most  effective  methods  for  their  final  product.  We  would 
assert  that  refined  methods  developed  using  an  early  usability  testing  model  must  be  more  effective 
than  prescribed  methods  tested  only  once  under  a  traditional  ISD  modH. 

•  Encourages  a  cyclical  process:  Dumas  and  Redish  (1993)  differentiate  between  two  types  of 
process  models:  assembly  line  and  cyclical  team  models.  In  the  assembly  line  process,  designers 
develop  their  products  in  successive  and  separate  phases.  There  is  a  dearth  of  feedback  between 
each  of  the  phases.  An  example  of  this  model  is  where  a  design  team  develops  a  product  during 
the  development  phase,  and  then,  the  marketing  department  (a  different  set  of  individuals)  evaluates 
th^  same  product.  There  is  limited  communication  between  these  two  groups  and  thus, 
assumptions  and  realizations  of  the  design  team  are  not  communicated  to  the  marketing 
department.  In  contrast,  the  cyclical  team  process,  a  design  team  performs  each  step  of  the  design 
process  (e.g.,  analysis,  development,  evaluation).  The  assumptions  and  realizations  of  the  design 
are  fully  communicated  to  each  member  on  the  design  team.  The  early  usability  testing  model  is 
explicitly  cyclical,  encouraging  both  communication  between  all  team  members  and  re- 
examination of  prior  decisions  on  each  repetition  of  the  cycle.  We  are  not  suggesting  that  all 
instructional  designers  who  follow  the  traditional  ISD  process  are  using  an  assembly  line  process. 
However,  the  traditional  ISD  model  presents  cyclical  development  as  a  desirable  option,  not  as  a 
requirement. 

•  Concentrates  on  the  human  element  in  the  process:  Early  usability  testing  seems  to  give  the 
eventual  user  of  an  instructional  product  a  consistent  "presence"  within  the  development  process. 
Once  designers  have  observed  real  people  use  even  an  approximation  of  their  product  at  the  earliest 
stages  of  conception,  they  tend  to  refer  back  to  those  people  throughout  the  project  to  visualize  the 
effect  that  future  design  changes  might  have  on  them. 

Conclusion 

Our  study  has  only  touched  the  surface  of  the  issues,  and  we  know  that  some  of  the  controversies  that 
attend  user -centered  design  accompany  its  introduction  into  ISD,  for  example,  the  question  of  cost/benefit 
analysis  for  the  additional  effort  spent  conducting  usability  sessions  and  debates  over  specific 
methodological  applications  for  one  type  of  project  and  another.  However,  our  experience  suggests  that  the 
possibilities  for  incorporating  user-centered  design  concepts,  and  specifically  usability  testing,  in  the  ISD 
process  deserve  consideration  and  further  study. 
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Abstract 


This  study  analyzed  new  forms  of  student  social  interaction  and  dialogue  within  asynchronous 
communications  of  six  middle  schools  and  six  high  school  participating  in  the  World  Forum.  In  the  World 
Forum,  students  discussed,  questioned,  and  debated  with  Arctic  explorers,  researchers,  World  Forum 
mentors,  and  peers  about  environmental  issues.  One  of  the  three  key  tasks,  Flash  Points,  generated  more 
lengthy  dialogue  than  other  techniques  (i.e.,  Arctic  Alerts  and  Questions  to  Explorers).  Analysis  of  the 
forms  of  assistance  provided  students  indicated  that  mentor  interactions  with  students  was  mainly  to  provide 
feedback,  question,  and  cognitiveiy  structure  the  lesson  or  activity;  minimal  instruction  or  modeling  of  how 
to  interact  occurred.  Few  student  questions  about  environmental  issues  or  exploratory  activities  were  of  an 
evaluative  or  analytical  nature,  but,  instead,  most  were  fact  or  knowledge-based.  Despite  the  minimal 
assistance  and  the  low-level  of  questioning,  student  role  taking  activities  within  their  environmental 
discussions  (here,  students  assumed  roles  of  famous  people  like  Professor  Stephen  Jay  Gould  and  Mr. 
Richard  Leakey)  enhanced  the  degree  of  perspective  taking  in  their  conversations.  Examining  the  interaction 
patterns,  forms  of  mentor  assistance,  level  of  questioning,  and  degree  of  perspective  taking  provided  new 
insights  into  the  impact  of  electronic  communication  on  student  learning,  but  additional  assessment 
techniques  are  still  needed. 

Introduction 

As  society  rides  the  crest  of  the  Information  Age  wave,  futuristic  predictions  become  pedestrian 
(Naisbitt,  1984;  Schrage,  1990;  Toffler,  1980).  In  theory,  students  and  teachers  soon  will  have  access  to 
"the  best"  information  regardless  of  "geography,  distance,  resources,  or  disability."  At  the  same  time  as 
these  new  technologies  advance  opportunities  for  student-student  distance  collaboration  and  dialogue,  many 
educators  are  turning  to  sociocultural  theories  like  Vygotsky's  (1986)  to  analyze  learning  in  a  social  context 
(Tharp  &  Gallimore,  1990;  Wells,  1994).  Disappointed  with  learning  theories  that  advocate  discrete  or 
inert  knowledge,  research  on  the  social  context  of  learning  lends  support  to  a  more  learner-centered  school 
pedagogy  (Alexander  &  Murphy,  1994;  The  Cognition  and  Technology  Group  at  Vanderbilt,  .991 ;  Duffy 
&  Jonassen,  1991;  Tharp,  1993).  The  collaboration,  negotiation,  questioning,  and  scaffolding  of  more 
"active"  learning  environments  is  drawing  attention  to  this  movement. 

This  study  offers  one  glimpse  of  what  how  these  trends  in  technology  and  learning  theory  might 
be  married.  Tools  for  writing  and  conversation,  for  instance,  are  elevating  many  school  activities  from 
silent,  solitary  acts,  to  one's  rich  in  student  discussion,  dialogue,  and  debate  (Bonk,  Medury,  &  Reynolds, 
1994).  However,  researchers  have  yet  to  make  significant  progress  regarding  how  cognitive  processes 
displayed  on  a  social  plane  such  as  electronic  mail  and  dialogue  eventually  become  internalized  by  the 
participants  as  independent  problem  solving  skill  (Wertsch,  1985).  Before  sociocultural  ideas  can  impact 
educational  reform,  new  approaches  are  needed  to  observe  and  measure  the  impact  of  intermentai  activities 
(e.g.,  student  collaboration  over  an  electronic  network)  on  intramental  processes  (Meloth  &  Deering,  1994; 
Wertsch,  1991). 

One  of  the  major  purposes  of  this  study  is  to  describe  and  analyze  student  interactions  while  using 
an  information  age  tool  (i.e.,  the  Internet).  Innovative  educational  activities  were  used  to  apprentice  young 
learners  into  actual  environmental  and  scientific  explorations.  As  a  result,  opportune  questions  regarding 
the  social  and  cognitive  benefits  of  new  interaction  patterns  must  now  be  raised  in  regards  to  types  of 
mentorship,  interaction  patterns,  levels  of  questioning,  and  the  interpersonal  impact  of  shared  meaning. 
Just  what  types  of  interaction  and  forms  of  learning  assistance  does  this  type  of  information  highway 
provide?  How  might  students  be  apprenticed  in  their  initial  attempts  to  drive  the  highway  and  form  new 
learning  collaborations?  These  are  a  couple  of  the  questions  that  emerge  when  learning  theories  are  married 
to  deliverable  technologies. 


A  Sociocultural  Lens  on-Line  Collaboration 

Among  the  ramifications  of  Information  Age  educational  tools  such  as  the  Internet  is  collaborative 
computing.  For  example,  students  might  send  messages  to  distant  people  or  distinguished  experts  on  a 
topic,  while  teachers  are  collaborating  with  their  colleagues.  Ishii  (1993)  asserted  that  the  increasingly  use 
of  telecommunications  will  build  an  "essential  foundation"  for  collaboration.  Collaborative  work  in  online 
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communities  becomes  an  effective  way  of  using  these  information  age  tools  (Harasim,  1989;  Ruopp,  Gal, 
Drayton,  &  Pfister,  1993).  If  predictions  hold,  students  will  be  empowered  and  proactive  in  their 
collaborative  learning  processes.  Riel  (1993),  for  instance,  argues  that  these  online  communities  will 
provide  a  global  education  for  students,  enabling  them  to  see  the  complexity  of  the  world  through  other 
viewpoints.  Through  collaborative  environments,  students  can  appreciate  the  complexity  of  issues  and 
consequences  of  their  actions.  According  to  Riel,  this  global  community  can  promote  positive  attitudes  and 
greater  global  awareness. 

Riel  (1985)  contends  that  these  new  possibilities  for  cooperative  learning  and  students'  interactions 
engage  students  in  a  process  of  creating  shared  meanings.  In  her  opinion,  online  networks  create  new 
possibilities  of  teaching  and  learning  by  "facilitating  group  communication  among  classrooms  and  teachers" 
(Riel,  1990)  and  fostering  collaborative  production  and  analysis  of  information  (Riel,  1992).  Bannon 
(1986)  similarly  notes  that  computer-mediated  communication  (CMC)  will  "help  to  create  new 
communities  of  people,  bound  by  a  shared  interest  in  a  topic  or  a  shared  background."  Hence,  the 
interaction  between  humans  and  computers  should  not  only  be  investigated  through  a  cognitive  lens,  but 
also  through  a  social  lens  (Brown,  1986;  Scott,  Cole,  &  Engel,  1992).  Brown  indicates  the  need  to  observe 
and  make  sense  of  the  social  interactions  of  users  who  interact  with  collaborative,  online  systems.  In  such 
an  analysis,  one  begins  to  distinguish  how  students  can  broaden  their  perspectives  through  global  education 
and  computer-mediated  communication  and  become  full  participants  in  it  (Riel,  1993). 

Theoretical  Framing  of  this  Sociocultural  Lens 

In  this  developmental  theory,  human  mental  functioning  is  inherently  situated  in  social 
interactional,  cultural,  institutional,  and  historical  context  (Wertsch,  1991).  Basically,  an  individual  derives 
meaning  from  the  tools  and  signs  mediating  his  social  and  individual  environment  within  the  phase  of 
development  Vygotsky  refers  to  as  the  zone  of  proximal  development  (ZPD)  (Salomon,  1988;  Wertsch, 
1991 ).  A  ZPD,  the  distance  between  a  child's  independent  problem  solving  level  and  the  that  obtained  under 
adult  guidance  or  in  collaboration  with  more  capable  peers,  might  be  evident  when  peers  can  teach  other 
students  about  his  or  her  particular  community  when  using  on-line  technologies.  Note  that  recent 
collaborative  writing  studies  lend  evidence  that  students'  internalize  the  scaffolding  of  more  capable  peers 
when  collaboratively  writing  (Daiute  and  Dalton,  1988)  as  well  as  the  cognitive  supports  or  prompts 
provided  by  computer  tools  (Salomon,  1988;  Zellermayer,  Salomon,  Globerson,  &  Givon,  1991). 
However,  Salomon  (1993)  cautions  that  intellectual  benefits  will  depend  on  the  extent  of  student 
"mindfulness"  or  engagement  when  collaborating  with  peers  using  technology. 

More  capable  peers  or  adult  guides  on  a  computer  network  may  lead  students  into  writing  or 
communication  tasks  in  which  they  might  not  typically  consider  or  address.  Likewise,  from  a  Piagetian 
standpoint,  debating  and  dialogue  with  peers  over  a  network  guides  and  challenges  learners  to  new  levels  of 
growth  and  understanding  (Clements  &  Nastasi,  1988).  When  confronted  by  alternative  views  and 
opinions,  cognitive  dissonance  or  disequilibrium  is  triggered,  thereby  causing  individuals  to  seek  additional 
information  to  resolve  to  conflict  (Piaget,  1963).  Both  Vygotskian  and  Piagetian  theory,  therefore,  seem  to 
have  foreshadowed  the  emergence  of  collaborative  technologies. 

Another  developmentalist,  Robert  Selman  (1980),  suggested  that  educators  need  to  devise  new 
ways  for  students  to  progress  beyond  their  egocentric  views  of  the  world.  His  construct,  perspective-taking 
or  social  cognition  (i.e,  "to  infer  another's  capabilities,  attributes,  expectations,  feelings  and  potential 
reactions,"  Selman,  1971 )  has  been  studied  in  a  variety  of  settings  during  the  past  thirty  years.  Similar  to 
Piaget's  (1963)  cognitive  developmental  theory,  Selman  (1976,  1980;  Selman  &Byrne,  1974)  has  outlined 
a  social  cognitive  developmental  model  with  five  distinct  stages.  Selman  proposed  that  children  gradually 
decenter  from  egocentric  points  of  view  to  the  ability  to  take  into  account  one  or  more  points  of  view 
simultaneously.  For  instance,  the  sharing  of  views  on  a  computer  screen  might  foster  meaning  making 
activities  and  interpersonal  understanding.  Such  shared  experiences  with  distant  peers  on  a  computer 
network  offers  a  window  into  how  points  of  view  diverge  and  a  channel  for  displaying  thern.  Clearly, 
situating  the  idea  of  role  play  within  an  electronic  collaborative  network,  enhances  the  opportunities  for 
developing  social  cognitive  skills. 
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Though  global  education  might  elevate  social  cognitive  skill  and  scaffold  the  learning  process, 
there  is  no  guarantee  that  collaboration  and  interaction  will  trigger  critical  reflection  on  one's  ideas  or 
enhance  interpersonal  understanding.  New  forms  of  mentoring  and  teaching  assistance  might  address  this 
dilemma  (Tharp  &  Gallimore,  1988).  But  how  might  one  assess  the  forms  of  this  teaching  assistance? 
How  might  social  interaction  and  discourse  be  altered  by  the  participant  structures  and  degree  of  status 
among  the  participants?  How  might  cognitive  engagement  be  augmented  by  the  task  structuring  and 
questioning  provided  by  electronic  mentors? 

Evidently,  scaffolding  and  feedback  is  not  restricted  to  one's  peers  and  classroom  teacher,  but  now 
guidance  comes  from  other  teachers,  consultants,  and  experts  who  apprentice  learning  in  these  new  forums. 
As  a  result,  the  "social  and  intellectual  connectivities"  (Harasim,  1989,  p.  39)  afforded  by  collaborative 
technologies  advance  both  the  educational  boundaries  and  concerns.  Expanding  the  frontier  of  education, 
naturally,  places  the  participants  in  uncharted  waters.  Regardless  of  the  terminology— coordinators,  coaches, 
advisors,  or  guides— the  apprenticing  opportunities  are  central  to  success  here.  Lehrer,  Erickson,  Love,  and 
Connell  (in  press)  asserted  that  the  notion  of  an  apprenticeship  might  translate  ccciocultural  principles  into 
"good"  instructional  practice.  In  a  related  article,  Lehrer  (1993)  contends  that  instructional  practices  within 
apprenticeship  environments,  therefore,  might  guide  students  into  the  learning  process,  promote  student 
design  of  knowledge,  multiple  representations  of  that  knowledge,  and  student-student  dialogue,  integration, 
and  presentation  of  that  knowledge  (Lehrer,  1993).  Rogoffs  (1991)  findings  that  adult-child  dyads  often 
foster  more  sophisticated  problem  solving  then  child-child  dyads  support  apprenticeships  or  "guided 
participation"  educational  activities. 

Guided  learning  activities  that  involve  students  in  challenging  and  internally  controllable  activities 
empower  learners  and  facilitate  their  active  construction  of  knowledge  (Wells  &  Chang-Wells,  1 992). 
According  to  Wells  and  Chang- Wells,  assisted  performance  might  further  draw  students  into  an  inquiring 
community  of  learners.  As  experts  in  the  community  assist  novice  learners  in  assuming  a  greater  portion 
of  the  responsibility  or  intellectual  burden  of  the  task,  they  begin  tc  appropriate  and  internalize  authentic 
cultural  practice.  Using  technology  to  build  communities  of  practice  and  apprentice  less  skilled  learners 
through  social  interaction  and  dialogue  is  now  possible  through  a  on-line  educational  network.  An  expert 
can  arrive  on  a  network  and  participate  with  learners  in  a  joint  learning  activity  wherein  the  expert  might 
provide  data,  demonstrations,  or  hypotheses.  Novice  learners,  in  contrast,  might  seek  additional 
information,  ask  questions,  or  offer  tentative  conclusions  and  recommendations.  Teachers  or  cognitive 
coaches  might  mediate  between  these  two  roles,  by  establishing  the  initial  goals  oi  tasks,  stating  necessary 
assumptions,  model  expected  thought  processes,  and  guiding  the  overall  learning  process  through  support, 
hints,  and  clues  (Collins,  Brown,  &  Newman,  1989). 

Since  the  seminal  article  on  the  emergence  of  cognitive  apprenticeships  in  education  by  Collins  et 
al.  (1989),  educators  have  sought  new  avenues  for  creating  mentor-apprentice  relationships  and  embedding 
the  learning  in  a  more  legitimate  social  context.  Instead  of  a  continuation  down  the  path  to  inert 
knowledge,  schools  are  searching  for  maps  to  more  realistic  learning  situations  and  avenues.  The  cognitive 
apprenticeship  reviewed  here  is  one  plausible  means  to  realize  these  goals. 

Method 

The  World  Forum  Activities 

The  World  Forum,  developed  by  the  University  of  Michigan,  is  a  key  component  of  the  World 
School  for  Adventure  Learning  currently  regulated  at  Indiana  University.  The  World  Forum  is  an  online 
asynchronous  telecommunications  project  designed  to  give  middle  and  high  school  students  the  opportunity 
to  interact  with  each  other  about  critical  environmental  and  social  issues.  This  particular  analysis  involved 
students  in  six  middle  schools  and  six  high  schools  participating  in  the  World  Forum. 


As  World  Forum  participants,  a  student  group  (with  two  or  three  students)  would  assume  the 
identity  of  a  famous  person;  for  example,  one  student  group  might  represent  Jacques  Cousteau  while 
another  student  group  might  represent  Margaret  Thatcher  (see  Appendix  A).  These  students  would  conduct 
background  research  on  their  particular  character  in  order  to  effectively  participate  in  the  World  Forum 
explorations  and  activities.  Additionally,  the  student  groups  were  expected  to  connect  online  with  the 
World  Forum  daily. 

There  were  four  main  participants  in  the  World  Forum:  (1)  World  Forum  organizers,  (2)  World 
Forum  mentors,  (3)  World  Forum  participants  (i.e.,  students)  and  (4)  Arctic  explorers.  The  World  Forum 
organizers  created  two  key  activities:  Flash  Points  and  Arctic  Alerts.  In  one  activity,  student  groups  were 
asked  to  respond  to  statements  called  Flash  Points,  in  similar  manner  the  character  they  were  assigned  to 
researched.  For  example  on  a  Likert  scale  (where  Strongly  agree  =  1  and  Strongly  disagree  =  5),  students 
would  voice  their  opinion  on  a  statement  such  as,  "The  pen  is  mightier  than  the  sword".  They  would  either 
agree  or  disagree  with  this  Flash  Point  and  state  their  reasons  for  their  particular  opinion. 

Arctic  Alerts,  the  second  instructional  activity,  were  news  reports  that  described  critical 
environmental  and  social  issues  in  the  Arctic  region.  One  Arctic  Alert  explored  the  issues  of  oil 
exploration  on  native  Eskimo's  land.  World  Forum  participants  responded  positively  or  negatively  to  these 
issues.  World  Forum  student  groups  also  received  reports  from  an  Arctic  expedition  conducted  in  the 
Spring  of  1994.  Student  groups  were  encouraged  to  pose  questions  to  the  Arctic  expedition  team.  Though 
participation  ultimately  varied,  students  were  instructed  to  log-in  to  the  World  Forum  each  day  and  voice 
their  responses  to  the  aforementioned  activities.  As  part  of  the  instructional  design  of  the  World  Forum, 
throughout  these  interactions  the  student  groups  were  assisted  by  World  Forum  mentors  who  questioned  and 
guided  the  student  groups'  understanding  of  these  environmental  issues. 

Student  interactions  via  the  World  Forum  lends  insight  on  the  process  of  effective  and  ineffective 
asynchronous  collaboration.  By  analyzing  these  processes,  asynchronous  on-line  communications  designs 
can  be  improved.  There  are  several  important  questions  to  ask  about  the  nature  of  this  type  of 
collaboration.  First  of  all,  what  kinds  of  social  spaces  do  participants  feel  comfortable  and  uncomfortable 
in?  Secondly,  what  kinds  of  communication  facilitate  a  common  sharing  of  knowledge?  Third,  how  are 
shared  meanings  constructed?  However,  since  this  paper  is  concentrating  on  the  mentor-student 
relationship,  it  is  important  to  ask  about  the  types  of  learning  assistance  specifically  provided  within  these 
virtual  cultures.  That  is,  how  do  these  teleapprenticeships  (Levin  et  al.,  1987;  Teles,  1993)  guide  their 
students/tutees  through  this  new  medium  and  successfully  communicate  with  other  peers. 

In  addressing  these  questions  and  concerns,  the  research  turns  from  hardware  connection  and 
information  access  issues  to  ones  regarding  how  mentor  assistance  and  scaffolding  is  embedded  in  e-mail 
dialogue.  The  analysis  tools  and  coding  schemes  chosen  here  offer  insight  into  the  how  social  interaction 
impacts  student  learning.  These  coding  schemes  were  designed  to  provide  insight  into  how  to  measure  the 
activity  setting  of  collaborative  computing.  The  four  variables  chosen  are  discussed  below. 

Data  Analysis  Procedures 

To  analyze  the  interactions  between  World  Forum  participants  and  their  mentors,  all  World  Forum 
discussion  strands  from  the  participants  and  mentors  but  not  explorers  were  subscribed  to.  After  the  World 
Forum  finished  approximately  eight  weeks  later,  this  data  was  downloaded  and  printed  out  for  further 
analyses.  As  noted  previously,  only  student  and  mentor  interchanges  were  saved  and  analyzed  here;  explorer 
notes  to  students  were  not  included  in  this  analysis.  By  having  students  assume  fictitious  identities,  all 
student  data  was  anonymous.  All  dialogue  data  was  time  and  date  stamp  with  school  indicators.  Though 
student  participants  were  asked  to  log  on  each  day,  more  activity  was  logged  during  the  first  few  weeks  than 
during  the  later  weeks. 

Four  analysis  techniques  emerged  after  surveying  the  three  key  social  interaction  activities  of  the 
World  Forum  mentioned  earlier,  namely,  Flash  Points,  Arctic  Alerts,  and  Student  Questions  to  Arctic 
explorers.  The  different  analysis  techniques  selected  here  were  based  on  the  particular  content  of  these  three 
activities.  Below,  we  briefly  describe  each  analysis  technique  and  how  we  applied  these  techniques  to  World 
Forum  interactions. 
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/.  Interaction  Tabulations  and  Analyses 

The  first  analyses  of  the  middle  and  high  school  data  entailed  counting  the  number  of  student 
generated  interactions  concerning  the  Flash  Points,  Arctic  Alerts,  and  Student  Questions  to  Explorers. 
After  taking  out  interaction  headers,  a  computer  count  of  the  total  number  of  words  composed  by  middle 
school  and  high  school  students  during  the  three  activities.  The  number  of  interactions  was  divided  into  the 
total  word  counts  to  calculate  words  per  interaction.  Counts  of  the  number  of  mentor-student  interactions 
also  were  noted. 

//.  Tharp  and  Gallimore's  six  means  of  assistance 

Tharp  and  Gallimore  (1988)  offer  insights  into  effective  means  of  facilitating  students 
understanding.  With  their  argument  that  schools  should  provide  assistance  for  the  entire  school 
community,  Tharp  and  Gallimore  identified  six  means  of  assisting  performance  including:  Modeling, 
Contingency  Management,  Feeding-back,  Instructing,  Questioning,  and  Cognitive  Structuring.  These  six 
techniques  of  assisting  understanding  can  contribute  in  facilitating  both  mentor  and  students  participation  in 
Collins  et.  al.'s  (1989)  cognitive  apprenticeship.  Thus,  interactions  between  mentors  and  World  Forum 
middle  school  and  high  school  were  coded  using  Tharp  and  Gallimore's  framework.  Questions  of  interest 
included  whether  the  assistance  varied  across  group  and  task  structure,  as  well  as,  over  time.  For  instance, 
what  types  of  learning  assistance  do  students  need?  And  does  this  mentors  assistance  vary  with  younger 
students  and  older  students  or  from  beginning  to  later  sessions?  To  begin  to  discern  answers,  percentages  of 
interactions  involving  each  assistance  mode  were  calculated  for  both  grade  levels  as  well  as  overall 
interactions. 

///.  Bloom's  taxonomy  of  educational  objectives 

Paralleling  the  forms  of  assistance,  was  a  concern  with  the  level  of  questioning  in  electronic  social 
interaction  and  discourse.  Though  decades  old,  Benjamin  Bloom's  (1956)  cognitive  taxonomy  from  low 
level  knowledge,  comprehension,  and  application  goals  to  high  level  analysis,  synthesis,  and  evaluation 
educational  objectives  was  useful  in  analysis  electronic  discourse.  Though  these  question  types  have  been 
used  during  the  past  four  decades  to  illustrate  the  lack  of  high-level  questioning  in  the  classroom,  perhaps 
electronic  interaction  in  a  community  of  discourse  would  elevate  observed  questioning  level.  Bloom's 
taxonomy  definition  helped  measure  the  levels  of  thinking  that  World  Forum  participants  used  when  asking 
questions  to  Arctic  explorers.  We  speculated  whether  engaging  in  the  other  World  Forum  activities  (i.e., 
Flash  Points  and  Arctic  Alerts),  students  would  gain  more  expertise  and  express  this  new  knowledge  with 
higher  level  questions.  Moreover,  would  students  internalize  and  exhibit  the  types  of  questions  asked  by 
their  World  Forum  mentors? 

IV.  Selman's  Perspective-Taking 

In  adopting  the  latter  four  of  Selman's  (1980)  perspective-taking  categories,  coded  interactions  with 
both  the  Flash  Points  and  the  Arctic  Alerts  with  Selman's  perspective-taking  coding  scheme.  The  primary 
question  asked  was  whether  students'  perspective-taking  would  be  enhanced  in  one  activity  rather  than 
another  one.  Would  students'  responses  to  Arctic  Alerts  encourage  higher  levels  of  perspective  taking  than 
Flash  Points?  Secondly,  we  asked  whether  older  students  would  exhibit  higher  levels  of  peispective-taking 
than  middle  school  students.  Of  course,  most  students  in  World  Forum  interactions  involved  role  playing 
from  the  perspective  of  a  famous  person;  hence,  participants  were  already  at  operating  above  initial 
egocentric  level. 


Results 

In  first,  taking  a  quantitative  look  at  the  data,  we  found  that  the  number  of  interactions  recorded  in 
World  Forum  (see  Table  #1),  both  middle  and  hig'i  school  students  wrote  a  similar  amount  of  messages 
(Flash  Points,  Arctic  Alert  responses,  and  Questions  to  Arctic  explorers).  The  Flash  Point  activity  was 
used  the  most.  It  is  interesting  to  note  that  high  school  students  used  much  more  words  per  average  in 
Flash  Points  and  their  Student  Questions  to  Explorers  than  middle  school  students  (i.e.,  7805  words  as 
opposed  to  4808  words  in  Flash  Points;  1030  words  as  opposed  to  625  words  in  Student  Questions  to 
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Explorers).  Surprisingly,  the  student  interaction  activity  decreased  during  the  eight  weeks  and  direct  student- 
student  interaction  throughout  the  project  was  minimal. 

Table  1  also  indicates  that  World  Forum  mentors  focused  the  Arctic  Alert  activity  on  middle 
school  students  (i.e.,  7  as  opposed  to  1),  whereas  their  Flash  Points  and  Student  Questions  to  Explorers 
provoked  more  high  school  student  interactions  (31  and  13  interactions,  respectively,  for  high  school 
students  compared  to  15  and  1  middle  school). 

World  Forum  mentor  interactions,  the  coding  scheme  based  on  Tharp  and  Gallimore's  forms  of 
learning  assistance  (see  Table  #2),  disclosed  mat  the  Questioning  technique  was  the  most  frequently  used 
with  novices.  Surprisingly,  one  of  the  six  forms  of  assistance,  namely  modeling,  was  totally  absent. 
Mentors  could  have  used  this  technique  to  promote  more  interactions  among  students.  For  instance,  in  the 
beginning  of  the  World  Forum,  modeled  interactions  could  be  demonstrated  to  middle  and  high  school 
students.  This  technique  would  lay  the  ground  work  on  how  to  properly  interact  with  other  World  Forum 
participants.  The  Contingency  Management  also  could  have  been  used  to  facilitate  this  interaction.  In  the 
beginning  of  the  World  Forum,  mentors  could  have  reminded  World  Forum  participants  that  they  should 
respond  to  their  counterparts.  By  "managing"  these  students,  there  could  have  possibly  been  more  peer 
interactions.  Overall,  World  Forum  mentors  did  a  good  job  of  asking  questions  and  providing  cognitive 
structure  to  the  task,  but  failed  to  provide  a  framework  of  how  students  should  properly  interact  with  one 
another. 

A  majority  of  both  middle  school  and  high  school  student  questioning  activities  to  the  Arctic 
explorers  (see  Table  #4)  appear  to  be  at  lower-level  thinking  on  Bloom's  taxonomy  (i.e.,  Knowledge  and 
Comprehension  levels).  There  was  no  apparent  transfer,  therefore,  from  engaging  in  the  other  World  Forum 
activities  (i.e.,  Flash  Points  and  Arctic  Alerts).  However,  since  there  were  only  thirty  questions,  this 
speculation  cannot  be  confirmed.  Again,  we  would  recommend  that  World  Forum  mentors  and  staff  model 
higher-level  questions  for  World  Forum  students  or  give  out  sample  questioning  sheets  to  guide  their  early 
interactions. 

Examining  students'  perspective-taking  abilities  (see  Table  #5),  we  found  that  both  World  Forum 
activities  (i.e.,  Flash  Points  and  Arctic  Alerts)  promoted  higher-level  perspective  taking.  Though  51%  of 
middle  school  students'  responses  to  Flash  Points  were  scored  a  "1"  (Subjective  perspective-taking),  almost 
20%  of  their  responses  were  a  "4"  (Societal  perspective-taking)  (see  Appendix  E  for  sample  of  each 
perspective-taking  level  within  Arctic  Aierts  responses).  High  school  students  fared  better  with  Flash 
Points;  26%  of  their  responses  were  a  "3"  (Third-person  perspective-taking)  and  31.4%  of  their  responses 
were  a  "4"  (Societal  perspective-taking).  For  Arctic  Alerts,  both  middle  and  high  school  students  had  higher 
perspective-taking  levels.  In  fact,  it  appears  that  Arctic  Alerts  promoted  more  "3"  perspective-taking  among 
both  groups  of  students  than  its  Flash  Point  counterpart.  In  either  case,  high  schools  were  interacting  at  a 
predictably  higher  level  of  perspective-taking  than  middle  school  youth. 

Conclusion  and  Implications 

The  World  Forum,  developed  by  the  University  of  Michigan,  simulated  an  on-line  discussion 
between  middle  and  high  school  students  about  critical  environmental  issues.  The  intention  of  this 
instructional  activity  was  for  students  to  assume  the  identities  of  famous  characters  and  communicate  with 
their  peers,  explorers,  and  mentors  about  these  environmental  issues.  Student  interactions  on  the  World 
Forum,  were  analyzed  using  four  different  analysis  methods,  namely,  frequency  of  responses  as  well  as 
analyses  derived  from  the  work  of  Tharp  and  Gallimore  (1988),  Bloom  ( 1 956),  and  Selman  (1971).  Results 
indicate  that  the  World  Forum  participants  assumed  higher  levels  of  perspective-taking,  but  lacked  any 
significant  amount  of  peer  collaboration.  World  Forum  participants  demonstrate  fairly  high  perspective- 
taking  abilities  while  engaging  in  Flash  Points  and  Arctic  Alerts.  The  longer  and  more  contemporary, 
Arctic  Alerts,  were  responded  to  at  a  slightly  more  higher  level.  Though  students  demonstrated  these 
higher-level  abilities,  especially  high  school  students,  they  did  not  transfer  this  knowledge  when  asking 
questions  to  Arctic  explorers.  Instead  their  questions  remained  primarily  at  the  Knowledge  and 
Comprehension  levels.  Combined  with  the  findings  on  mentor  assistance,  our  results  indicate  that  proper 
mentoring  for  students  engaging  in  computer-mediated  communications,  should  extend  beyond  mentors 
providing  tasks  and  questions  to  better  organizational  the  task  and  model  appropriate  responses. 


The  analyses  also  corresponded  with  expectations  of  developmental  differences  from  middle  to  high 
school.  High  school  students,  on  average,  wrote  more  lengthy  responses  than  middle  school  youth.  More 
specifically,  grade  level  differences  indicate  that  the  longer  and  more  concrete  Arctic  Alerts  may  provoke 
more  discussion  among  middle  school  students  than  high  school  students.  The  fact  that  the  more  thought- 
provoking  Flash  Points  and  Student  Questions  to  Explorers  were  utilized  by  older  students  corresponds  with 
their  ability  to  reason  at  high  levels  than  younger  students  (Piaget,  1963).  In  terms  of  direct  form  of 
assistance,  mentors  provided  equivalent  amounts  of  feedback,  questioning,  instructing,  and  cognitively 
structuring  the  tasks  to  both  age  groups,  but  used  behavioral  management  slightly  more  often  with  younger 
students.  In  terms  of  level  of  questioning,  too  few  questions  were  asked  to  infer  any  developmental 
differences.  Finally,  the  perspective  taking  analyses  indicated  that  older  students  responded  to  Flash  Points 
and  Arctic  Alerts  at  more  often  at  Selman's  upper  levels  of  social  cognitive  ability  or  judgment  than  the 
early  adolescent  students  in  this  study.  As  predicted  (Muuss,  1988),  as  children  mature,  role  taking 
activities  or  the  ability  to  make  inferences  about  the  other  people's  capabilities  and  potential  reactions 
become  easier.  Overall,  these  codings  of  electronic  data,  though  tentative,  begin  to  detail  why  innovative 
social  interaction  formats  may  need  to  consider  student  developmental  level. 

The  World  Forum  is  promoted  as  an  exemplar  tool  for  students  to  engage  in  critical  environmental 
and  social  issues  from  another  person's  perspective.  It  is  a  means  to  marry  simultaneous  advances  in  theory 
and  technology.  Through  programs  like  the  World  Forum,  electronic  collaboration  between  students, 
teachers,  and  experts  is  beginning  to  dramatically  impact  educational  settings  by  fostering  more  learner- 
centered  activities  and  opportunities  to  socially  share  meaning.  In  using  the  Internet,  several  novel,  on-line 
educational  activities  were  created  to  communicate  with  peers,  assume  roles,  and  create  meaning  and  share 
opinions  with  researchers  and  explorers.  As  more  apprenticing  opportunities  emerge  from  computer- 
mediated  communication  environments,  additional  understandings  are  needed  of  how  the  intermental 
processes  among  participants  become  internalized  as  independent  problem  solving  processes.  Extensive 
research  is  needed  to  examine  how  best  to  design  similar  instructional  tools. 

Future  research  ideas  spin-off  from  each  limitation  of  this  study.  First  of  all,  we  did  not 
manipulate  the  interactions  among  participants,  but  simply  analyzed  the  social  interaction  and  dialogue  of 
students  and  schools  within  an  existing  program.  In  a  follow-up  effort,  researchers  might  want  to  suggest 
tasks  or  collaboration  patterns  that  participants  will  use.  Variations  in  the  role  taking  or  questioning  of 
explorers,  for  instance,  may  enhance  the  findings  found  here.  At  the  same  time,  additional  training  of 
mentors  or  explorers  may  scaffold  their  ability  to  model  higher  levels  of  thinking,  questioning,  and 
perspective  taking  than  witnessed  here.  Peer  training  also  may  prove  fruitful,  since  few  of  the  student 
interactions  noted  here  directly  addressed  peers  at  participating  schools.  Of  course,  other  researchers  might 
want  to  perform  similar  analyses  on  the  piece  of  data  we  were  missing,  namely  the  explorer  responses  and 
written  notes  to  the  student  participants.  Third,  the  researchers  here  simply  downloaded  existing  data 
without  verifying  their  interpretations  with  the  teachers  or  students.  In  contrast  to  this  approach, 
researchers  might  want  to  use  social  interaction  data  as  an  instructional  tool  that  students  and  teachers  would 
evaluate  both  before  and  after  it  was  coded  for  form  of  assistance,  questioning,  and  perspective  taking.  In 
fact,  such  dialogue  transcripts  might  be  useful  for  preservice  teacher  education  programs  struggling  with 
how  to  concretize  sociocultural  concepts  such  as  scaffolding,  zones  of  proximal  development,  and  assisted 
learning.  Of  course,  though  student  competence  is  the  ultimate  goal,  we  did  not  compare  student 
performance  or  address  learning  transfer  to  other  classroom  activities. 

Despite  these  limitations,  some  of  the  tasks  and  approaches  used  here  help  clarify  the  impact  of 
social  interaction  on  student  learning.  Human  learning  and  development  theory  was  used  as  a  guide  in 
assessing  the  impact  of  new  technologies  and  interaction  structures.  Certainly  these  coding  schemes  for 
analyzing  questions  asked,  help  provided,  and  perspectives  taken  can  be  further  refined  or  altered  for  other 
collaboration  formats.  Indeed,  there  are  additional  variables,  coding  schemes,  technologies,  and  instructional 
design  models  to  consider.  Though  network  and  on-line  education  technologies  are  changing  faster  than 
research  studies  can  be  designed,  in  this  study,  we  used  four  assessment  techniques  that  were  specific  to  the 
social  interactions  and  mentorship  of  the  World  Forum.  It  is  a  start.  Future  activities  might  explore 
additional  World  Forum  data  and  fine-tune  these  initial  social  interaction  coding  schemes  for  collaborative 
learning  and  e-  mail  dialogue.  Nevertheless,  we  are  beginning  to  peer  into  how  social  interaction  and 
discourse  influence  electronic  group  activities  and  resulting  cognitive  change. 
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Table  1. 

Student  Interaction  by  Activity  in  World  Forum  Dialogue 


Interactions/ Activity 


Flash 

middle  school: 
Flash 

high  school: 
Arctic 

middle  school: 
Arctic 

high  school: 

Questions 
middle  school: 


#of 

messages 


101 


Questions 
high  school: 


123 

56 

45 

16 

13 


#of 
words 


4808 
7505 
4369 
3898 
625 
1030 


Average  # 
words 


39 


77 
78 
86 
39 
79 


#of 

mentor/student 
interactions 


15 


31 


Table  2 

Forms  of  Teaching  Assistance  in  World  Forum  Dialogue 

(Note:  usually,  there  were  more  than  one  mode  of  assistance  per  mentor-student  interaction;  hence,  there  are 
more  assistance  techniques  than  interactions.  Also,  modeling  was  not  displayed  by  World  Forum  mentors.) 


Flash 

middle  school: 
Flash 

high  school: 
Arctic 

middle  school: 
Arctic 

high  school: 

Questions 
middle  school 

Questions 
high  school: 


Fdbk 

2 


O'ing     Cog.  Struct. 


25 


15 

3 


Instruct. 

3 


Model    Cont.  Manag. 

2 


0 


12 
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Table  3. 

Forms  of  Teaching  Assistance  in  World  Forum  Dialogue 

Middle  School  Total:  35  Mentor-Student  Interactions 
Feedback-  20% 
Questioning-  40% 
Cognitive  Structuring-  20% 
Instructing-  8.5% 
Modeling-  0.0% 

Contingency  Management-  11.4% 

High  School  Total:  87  Mentor-Student  Interactions 
Feedback-  19.5% 
Questioning-  43.6% 
Cognitive  Structuring-  24.1% 
Instructing-  12.6% 
Modeling-  0.0% 

Contingency  Management-  0.0% 

Overall  Total:  122  Mentor-Student  Interactions 
Feedback-  19.6% 
Questioning-  42.6% 
Cognitive  Structuring-  22.9% 
Instructing-  1 1 .4% 
Modeling-  0.0% 

Contingency  Management-  3.2% 


Table  4. 

Level  of  Student  Questions  to  Arctic  Explorers  in  World  Forum  Dialogue 

Know.   Compre.  Applic.  Anal.  Synth.  Eval.  Total: 

Middle  School:  5  6  0  1  0  4  16 

High  School:  7  5  0  1  1  0  14 

Overall:  12  11  0  2  1  4  30 

40%  36.7%  0.0%  6.7%  3.3%  13.3% 


Appendix  A 

Sample  of  characters  role  played  by  Some  World  Forum  participants 


Green  Delegation 

Professor  Stephen  Jay  Gould 
Monsieur  Jacques  Cousteau 
Senor  Roberto  Clemente 
Senora  Eva  Peron 
Professor  Sissela  Bok 
Mr.  E.  F.  Schumacher 


Purple  Delegation 

President  Lech  Walesa 

Ms.  Petra  Kelly 

Pope  John  Paul  II 

Prime  Min.  Kazimiera  Prunskiene 

Mr.  Ernest  Hemingway 

Mr.  Stephen  Biko 
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What  is  PisfmsxiSSl 

In  second  language  education  circles  for  the  past  ten  to  15  years,  the  word  "proficiency"  has 
generated  constable  debate  and  controversy.  The  word  does  not  simply  represent  or  sta  e  of 

being  proficient,"  as  Webster  (1993,  p.  93.)  would  have  it,  but  instead  has  become  a  syno W  e 
proficiency  movement  or  the  shift  toward  proficiency-oriented  instruction  (Birckbichler  &  Corl  1993? 

JlmTniTvT"'  u  inS!rUCt,i0n      31  thC  hCart  °f  thC  Paradi8m  shift  the  seco»d  language  ' 

community  has  undergone  that  closely  mirrors  general  trends  in  education  toward  holism  and 
constructivism. 

For  the  second  language  student,  proficiency  is  a  measure  of  what  tasks  can  be  accomplished  with 
acquired  language  and  how  well  they  can  be  accomplished.  The  task  can  be  registering  for  a  hotel  room  or 
ordering  a  meal  m  a  restaurant,  and  the  "how  well"  is  not  only  receiving  the  room  and  meal  intended  but 
a  so  understanding  associated  cultural  and  sociolinguistic  connotations  of  these  communicatee  Tel 
Such  an  accomplishment  may  seem  the  logical  outcome  of  any  second  language  program,  however  this 
was  not  the  case  prior  to  the  adoption  of  proficiency-oriented  instruction  nowever,  this 

,h  .  Bfr    ,M°St  P,r0gramS  dating  back  at  Ieast  10  or  s°  ycars-  (^d  actually  many  still  today)  focused  more  on 
5£  E TZnl     gUT  n0tHneC"Sarily  °n  Wtat       can  do  with      The  teaming  methods  employed 
paralleled  those  generally  used  ,n  education  at  the  time:  reductionistic,  structured  approaches  emphasizing 

^t^^^S. (1984)  described  them- 3  Wiage  of  ,inguistic ™  H£5S? 

The  audiolingual  method  was  a  common  technique.  Students  repeated,  and  committed  to  rote 
rTSuVM?    1  Wh   /       Phrase^- A"dioIingualism  closely  paralleled  the  instructional  systems  design 
S  i I  I?  m        °f  Programmed  instruction.  The  methods  were  adopted  from  B.F.  Skinner's  theoretical 
work  in  behaviorism  and  systems  approaches.  Programmed  instruction  reduced  tasks  to  small  components 
provided  earners  frequent  feedback,  and  defined  specific  measurable  objectives  which  could  be  eadHy 
assessed  (Seels,  1989).  This  method  was  widely  used  in  military  training  which,  in  turn,  SgXantly 
influenced  the  civilian  foreign  language  community. 

The  structured  approaches  revolved  around  the  linguistic  elements,  specifically  grammatical 
phono  og.cal  and  syntactic  features  of  language-elements  readily  and  reliably  defined.  Evaluation  was 
yp.cally  accomplished  using  discrete-point  tests  with  completion  or  multiple-choice  questions  on  grammar 
(L.skin-Gasparro,  1984).  While  such  methods  allowed  inexperienced  teachers  to  teach  large  numbL  of 
students,  students  often  were  unable  to  transfer  knowledge  outside  the  classroom;  in  a  foreign  language 
sense,  they  were  functionally  illiterate.  °  6 

In  the  late  1960s  an  evolution  toward  communicative  approaches  began.  The  objectives  of  these 
approaches  are  to  target  a  task,  such  as  obtaining  a  hotel  room  or  ordering  a  meal,  and  gaining  the  level  of 
communicative  pro fic.ency  necessary  to  successfully  complete  the  task.  This  paradigm  shift  necessitated 
specification  of  functional  proficiency,  a  job  undertaken  by  the  American  Council  on  the  Teaching  of 
Sr  LangUageSf  (^CT!L)  and  the  President's  Commission  on  Foreign  Language  and  International 
Studies.  Results  of  this  effort  were  the  ACTFL  Proficiency  Guidelines,  published  initially  in  1982  and 
revised  in  1 986,  and  the  adoption  of  proficiency-oriented  methods  on  a  broader  scale. 

The  ACTFL  guidelines  delineate  functional  proficiency  in  five  areas  (speaking,  writing,  reading 
listening,  and  culture)  at  five  levels  (novice,  intermediate,  advanced,  advanced  plus,  and  superior)  with  the 
novice  and  intermediate  levels  further  categorized  as  low,  medium  or  high.  The  example  below  is  the 
guideline  for  speaking,  intermediate-mid  level: 

Able  to  satisfy  some  survival  needs  and  some  limited  social  demands.  Is  able  to 
formulate  some  questions  when  asked  to  do  so.  Vocabulary  permits  discussion  of  topics 
beyond  basic  survival  needs  such  as  personal  history  and  leisure  time  activities.  Some 
evidence  of  grammatical  accuracy  in  basic  constructions,  for  example,  subject  verb 
agreement,  noun-adjective  agreement,  some  notion  of  inflection. 

The  guidelines  are  intended  to  be  descriptive,  but  their  inherent  hierarchical  structure  often  encourages 
prescriptive  use.  Higgs  (1984)  refers  to  the  guidelines  as  "the  organizing  principle"  (p.4)  and  Liskin- 
Gasparra  (1984)  identifies  the  guidelines  as  "a  graduated  sequence  of  steps  that  can  be  used  to  structure  a 
foreign  language  program"  (p.  1 1 ).  The  guidelines  are  not  the  driving  force  behind  the  proficiency 
movement,  but  a  component  that  has  simplified  the  process  of  operationalizing  proficiency-oriented 
methods.  J 

This  is  a  paradigm  shift  in  the  true  sense  of  the  word,  as  communication  has  become  the  focus  of 
language  instruction  which  is  where,  in  my  opinion,  it  should  belong.  However,  as  previously  mentioned, 
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not  everyone  has  adopted  proficiency  methods;  many  programs  and  individual  teachers  still  focus  on 
grammar  translation. 

Applying  Instructional  Systems  Design  to  a 
Proficiency  Oriented  Foreign  Language  Program 

Manv  in  the  foreign  language  community  have  dismissed  instructional  systems  design  (ISD)  as 
being  able     mak  a  contribution  totteir  field.  This  may  stem  from  the  fact  that  ISC Ms  rooted I  deeply  ,n 

nne'  work  and  behaviorism,  and  that  it  may  not  be  "politically  correct"  to  hold  behavior  >  t 
pMosophbs  However,  for  a  couple  of  reasons,  ISD  can  be  of  great  value  in  designing  a  language 

Pr°gram  First  is  the  unfolding  of  the  proficiency  movement.  It  has  often  been  left  up  to  the  individual 
instructor  to  ncomorate  the  tenets  of  proficiency  into  instruction.  The  mdtv.dual  instructor  car,  use 
mstructor  to  .ncorporaie  i  v  election,  application  and  optimization  of  instructional  goals  and 

second  language  pTgrtrS.  These  are  guidelines  for  anyone  working  to  develop  a  anguage  .^donal 
oro.  am  which  can  include  an  instructional  designer  working  with  a  subject  matter  expert  (often  the 
lingSrtnguage  teacher),  or  the  second  language  teacher  as  well.  Many  of  these  guidelines  are  J-t  bas.c 
omm  n  sensf ,  but  following  them  without  skipping  any  of  the  steps  is  a  way  to  improve  an  ex  t  ng 
program  or  develop  a  new  one.  T*e  guidelines  closely  mirror  the  steps  in  the  ISD  process  the  steps, 
deffned  by  the  Dick  and  Carey  (1990)  model,  and  how  they  are  supported  by  each  guideline,  follow. 
Lastly,  these  guidelines  are  primarily  for  the  instructor-led  classroom. 

fi|  .,,  |inr  ,    rjge  to  ATFI  yp&skim  guidelines.  Analyzing  and  identifying  instructional  goals  is 

t  a  I       Arm  nrnfic  encv  guidelines  They  delineate  functional  orientation  and  specific  accuracy 

^J^^SSSSSm  b*W  ^  -d  cuhu- serve  as  goals  for  courses 

T^yteotn  oTer^ked,  but  a  good  deal  of  effort  went  into  their  development,  and  they  can  provide  a 
QnliH  reference  Doint  for  the  first  three  phases  of  the  ISD  process. 

S°lld  K{T^T^  IimSoMl  Goals  -  The  guidelines  provide  a  ready  made  set  of  specific 

spedfic  contoni  arJof  instruction,  such  as  an  English  for  Special  Purposes  program  that  teaches  how  to 
^^^£^^2^^^-  Most  language  use  requires  gaining  some  verba, 

shodd  fSus  oS  type  of  communicative  skill  in  which  proficiency  is  desired,  and  on  the  specific  content 

°f      '"I  °n  .^p-  Identify  Entry  Mmi™  .nH  Characteristics  -  The  guidelines  state  specifically  what 
Earners  should  be  able  to  do  after  instruction.  Omaggio  (1984)  stresses  the  value  of  the  guidelines 
hierarchical  form: 

The  guidelines  are  (also)  idea  y  suited  for  organizing  instruction  because  they 
are  regressive  Knowing  what  competencies  lie  at  the  next  level  will  help  us  sequence 
mate  rial  tc conform  to  nLral  developmental  patterns  in  adult  second-language  learners 
S3  prepare i  for  making  progress.  The  descriptions  will  also  allow  us  keep  ,n  mind 
the  ultimate  goal(s)  learners  hope  to  achieve,  (p.  44). 

The  hierarchical  nature  of  the  guidelines  indicate  what  skills  should  be  achieved  at  Novice  (low,  medium. 
EThwES ^  ate  (low  medium  high),  Advanced/advanced-plus,  and  superior.  Prior  to  des.gmng 
Suct^nTa  par, icula  level,  the  previous  level  should  be  achieved  as  entry  level  behavior. 
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Guideline  2  -Write  Specific  Performance  Objectives  and  Criterion  Referenced  Test  Items  Before.  Developing 
Instruction:  Matching  test  items  with  instructional  goals  has  been  one  of  the  most  problematic  areas  of 
language  programs.   Basic  1SD  practice,  however,  can  help  alleviate  this  problem,  particularly  if  the  goals 
have  been  clearly  delineated  as  discussed  above. 

ISP  Step:  Write  Performance  Objectives  -  Using  the  skills  identified  in  the  instructional  analysis, 
performance  objectives  that  indicate  the  conditions  and  criteria  that  are  required  of  the  learner  should  be 
prepared.  These  conditions  should  include  what  information  is  to  be  given  to  the  student  beforehand,  what 
type  of  skills  (oral,  written,  etc.)  will  be  expected,  and  specific  criterion  on  the  structural  accuracy  required. 
For  example:  "When  given  a  written  scenario  in  which  two  people  interact,  the  learner  will  write  a  Russian 
greeting  and  form  of  address  of  the  appropriate  formality.  Items  must  be  spelled  correctly."  These  items 
again  closely  reflect  the  ACTFL  guidelines,  but  will  make  it  clear  to  the  learner  what  performance  is 
expected,  and  make  it  easier  for  the  instructor  to  develop  the  instruction  and  evaluate  the  student. 

ISP  Step:  Develop  Test  Items  based  on  the  Objectives  -  Student  evaluation  should  be  criterion 
referenced,  though  there  is  a  tendency  to  compare  student  performance. 

Also  on  testing,  Liskin-Gasparro  (1984)  recommends  specific  assessment  objectives  based  on  the 
three  elements  of  the  guidelines,  which  define  functions  the  speaker  can  express,  the  content  of  those 
functions,  and  the  degree  of  accuracy.  The  last  function,  accuracy,  can  be  evaluated  in  typical  discrete  point 
fashion,  however,  recall  that  the  proficiency  movement  emphasizes  moving  instruction  away  from 
linguistic  structure.  The  focus  is  thus  on  the  first  two  areas,  function  and  content. 

The  most  obvious  method  to  test  an  acquirer's  proficiency  or  communicative  competence  is  with 
an  oral  interview,  which  measures  both  aural  comprehension  and  speaking  ability.  Krashen  and  Terrell 
(1983)  recognize  the  value  of  lengthy  oral  interviews,  however,  they  also  recognize  that  conducting  such 
interviews  with  large  classes  may  not  be  feasible.  A  possible  solution  is  testing  supported  by  CALL, 
which  can  reduce  the  interface  time  between  student  and  instructor.  Students  viewing  video  or  text  can  be 
tested  for  aural  and  reading  comprehension,  and  can  also  be  tested  for  writing  in  the  target  language  by 
producing  summaries.  The  instructor's  time  with  each  student  is  significantly  reduced  as  only  speaking 
abilities  need  to  be  evaluated  during  oral  interviews. 

Guideline  3  -  Make  the  Classroom  Learner-Centered: 

ISP  Step:  Develop  Instructional  Strategy.  The  Proficiency  means  that  students  must  practice  the 
skills  that  they  are  to  attain.  This  means  that  they,  not  the  instructor,  should  be  the  focus  of  any  language 
program.  A  variety  of  activities  that  require  student  performance,  small  group  work  that  increases  their 
opportunities  and  reduces  anxiety,  and  making  lots  of  input,  at  the  appropriate  level,  available,  are  strategies 
to  be  considered. 

An  argument  may  be  made  that  the  more  learner-centered  the  classroom,  the  weaker  the 
notion  of  controlled,  criterion  based  objectives.  Learner-centered  does  not  necessarily  mean  that  learners  can 
go  off  in  any  direction  they  see  fit,  but  instead  means  that  they  should  be  active  participants.  Students 
listening  to  a  dialog  between  the  teacher  and  one  other  student  should  be  minimized.  Whi'e  an  acceptable 
balance  must  be  found,  the  more  the  students  participate,  the  better. 

Guideline  4  -  Integrate  Instructional  Materials  and  Media: 

ISP  Step:  Pevelop  and  Select  Instructional  Materials.   Effective  use  of  materials  in  a 
proficiency-oriented  environment  is  another  significant  problem  facing  language  educators  today.  The 
quantity  and  quality  of  available  resources,  particularly  those  that  are  technology  based,  vary  significantly 
between  languages. 

Selection  of  materials  should  begin  with  a  comprehensive  review  of  instructional  goals  (which 
level  of  proficiency),  the  strategies  selected,  and  all  materials  available  for  the  language  selected.  Materials 
that  support  proficiency-based  instruction,  such  as  newspapers,  television  broadcasts,  and  films,  should  be 
considered  as  they  provide  input  variety  and  authenticity.  Textbook  use  should  be  limited  to  those  that 
have  evolved  from  grammar-based  to  functional/notional-based,  as  available.  Second  language  teachers,  in  a 
survey  by  Birckbichler  &  Corl  (1993),  cited  "goals  stated  in  terms  of  communicative  functions"  as  the  most 
critical  factor  in  textbook  selection.  Other  important  factors  noted  in  the  survey  included  varied  activities, 
contextual ized  exercises,  and  adherence  to  ACTFL  proficiency  guidelines  for  topics  and  sequencing. 
Instruction  often  closely  adheres  to  specific  texts,  which  despite  the  proficiency  movement  are  linguistic 
structure  oriented  (Birckbichler  &  Corl,  1993),  and  do  not  allow  for  any  supplemental  activities. 

Due  to  the  flood  of  new  technology  based  materials,  primarily  CALL  and  IVI,  particular  care 
should  be  taken  when  selecting  and  incorporating  them  into  a  program.  The  issues  to  consider  include  what 
levels  of  language  acquisition  are  best  served  by  CALL  or  IVD,  determining  the  correct  mix  of  language  lab 


and  classroom  time,  and  obtaining  software  and  materials  matching  curricula.  The  approach  generally  used 
in  second  language,  as  with  academic  fields  outside  of  second  language,  has  been  to  use  CALL/IVD 
applications  to  fill  gaps  in  curricula.  Integration  must  begin  with  strategically  placing  CALL/IVD  where  it 
is  most  efficient.  Video  segments  and  hypertext  exercises  must  be  carefully  matched  to  course  objectives, 
student  abilities  and  interests,  and  to  other  classroom  activities. 

A  basic  approach  is  to  use  the  level  of  the  language  acquirer  to  determine  how  much  technology  to 
use.  In  the  early  stages  of  language  instruction,  in  which  the  lexicon  must  be  the  focus  (Higgs,  1984), 
along  with  orthographies  in  some  languages,  CALL/IVD  applications  serve  to  link  lists  of  vocabulary  to 
context.  Hypertext  and  hypermedia  incorporate  embedded  strategies  not  typically  utilized  in  other 
disciplines,  such  as  chunking/clustering  and  frequent  short  periods  of  review  (McCoy  &  Wieble,  1983). 
They  can  also  provide  appropriate  sequencing  and  short  video  clips  to  provide  context.  Instructors  need  not 
rely  strictly  on  the  CALL/IVD  application.  Providing  vocabulary  lists  and  themes  in  advance  and  follow- 
up  by  shaping  classroom  activities  around  lab  work  will  enhance  acquisition. 

The  early  stages  require  the  greatest  degree  of  independent  effort  as  insufficient  lexical 
development  precludes  communicative  activity.  Otto  (1989)  believes  this  is  where  the  concentration  of 
CALL/IVD  applications  should  lie,  but  often  does  not.  Krashen  and  Terrell  (1983)  identify  a  period  early  in 
the  acquisition  process,  the  silent  period,  in  which  acquirers  say  very  little  except  for  memorized  whole 
sentences.  During  this  period  they  are  building  the  confidence  and  the  lexicon  necessary  to  produce 
language.  Larger  proportions  of  time  devoted  to  independent  CALL/IVD  activity  during  the  silent  period 
help  focus  on  aural  activity  and  reduce  anxiety  producing  oral  production.  Independent  activity  in  the  early 
stages  will  also  improve  attitudinal  factors. 

At  intermediate  levels,  increasing  the  quantity  and  quality  of  input  to  support  expansion  of 
vocabulary  is  the  focus  of  strategy  when  designing  instruction.  Dick  and  Carey's  ( 1 990)  strategy  of 
organization  is  most  appropriate  at  this  level  in  a  proficiency-  based  curriculum.  Presenting  vocabulary  in 
functional  or  topical  subsets  creates  a  framework  for  classroom  activity.  For  example,  with  the  theme  of 
ordering  a  meal  in  a  restaurant,  food  can  be  a  subset  around  which  numerous  activities  can  be  developed. 

Interactive  programs  should  be  combined  with  paired  or  small  group  work,  with  functionally  based 
content.  CALL/IVD  applications  play  an  obvious  role  at  this  level,  but  they  cannot  stand  alone.  Follow- 
up  with  classroom  discussion  of  the  events  of  the  video,  strictly  in  the  target  language,  will  assist  in  long 
term  retention  and  encoding.    Intermediate  levels  should  also  provide  opportunity  to  demonstrate 
proficiency  with  simulations  and  questioning  about  similar  personal  experience. 

Advanced  levels  of  language  study  require  a  lower  proportion  of  CALL/IVD  centered  activity  due  to 
the  level  of  technology  presently  available.  As  real  interactivity  cannot  occur,  the  genuinely 
communicative  work  required  at  this  level  must  be  left  to  the  classroom.  Input  is  still  crucial,  however, 
and  video,  news  broadcasts  and  printed  media  still  serve  a  substantial  role. 

Guideline  5  -  Evaluate  Your  Program:  Both  formative  and  summative  evaluation  can  improve  the 
effectiveness  of  second  language  instruction. 

ISP  Step:  Conduct  Formative  Evaluation  -  Formative  program  assessment  should  be  conducted 
continuously,  modifying  methods  and  activities,  updating  materials  and  adding  new  materials  that  will 
maintain  motivation.  Brierly  &  Kemble  (1991),  recommend  establishing  a  lifecycle  that  continuously 
analyzes  instructional  systems  to  determine  if  curricular,  teacher  and  student  requirements  arc  being  met. 

An  approach  modeling  Dick  and  Carey's  (1990)  three  phases  of  formative  evaluation  are  also 
useful.  One-to-one  and  small  group  evaluations  can  insure  that  instruction  is  interesting  and  motivational. 
The  third  phase,  the  field  trial,  in  which  students  are  placed  in  realistic  situation,  mirrors  the  goals  of 
proficiency  orientation.  Setting  up  situations  in  which  students  practice  language  will  assess  both  the 
student's  acquisition  and  the  effectiveness  of  the  instructional  program 

ISP  Step:  Conduct  Summative  Evaluation  -  Summative  evaluation  may  prove  problematic  as  an 
outside  evaluator,  normally  an  instructional  designer,  will  likely  not  have  acquired  the  language  being 
instructed.  This  lack  of  knowledge  complicates  the  process  as  the  evaluator  is  not  able  to  judge  the 
effectiveness  of  instruction.  In  such  cases,  the  evaluator  must  work  closely  with  the  designer  and  the 
teacher  to  determine  deficiencies  in  outcomes. 
Summary 

These  basic  guidelines  can  provide  the  language  program  developer  or  teacher  with  enough 
knowledge  of  basic  ISD  methodologies  to  enhance  program  development.  As  previously  stated,  many  of 
the  guidelines  are  just  basic  common  sense,  but  following  them  without  skipping  any  of  the  steps  is  a  way 
to  improve  an  existing  program  or  develop  a  new  one.  Using  a  systematic  and  systems  process  will 
improve  the  chances  that  established  goals  can  be  successfully  reached. 
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Abstract 


This  paper  discusses  the  importance  of  considering  the  social  context  in  which  an  instructional  product  will 
be  used  during  the  development  process.  The  authors  contend  that  traditional  instructional  product 
development  models  are  inadequate  because  they  ignore  social  context.  The  use  of  inadequate  traditional 
design  models  results  in  the  development  of  instructional  products  that  may  be  instructionally  effective  but 
that  are  not  desirable  to  potential  adopters.  Two  emerging  theories,  Adoption  Analysis  and  User -Oriented 
Instructional  Development,  are  presented  as  tools  that  instructional  developers  can  employ  in  order  to 
increase  the  adoption  of  their  products.  The  paper  describes  the  importance  of  incorporating  the  emerging 
theories  into  existing  instructional  development  models.  The  authors  conclude  by  calling  for  a  new  vision 
of  instructional  development  in  which  the  success  of  an  instructional  product  is  measured  by  its  successful 
adoption  just  as  much  as  success  is  now  measured  by  its  instructional  effectiveness. 
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Adoption  Analysis  and  User  Centered  Instructional  Design 


Technology  is  a  social  phenomenon.  The  design,  development,  adoption  and  diffusion  of 
technology  are  inherently  social  processes.  Technology  is  developed  by  people  in  a  particular  environment 
and  culture  and  is  intended  to  be  used  by  people  in  a  particular  environment  and  culture.  As  Howard  Segal 
writes  in  his  book  Future  Imperfect  (1994),  "all  structures  and  machines,  primitive  or  sophisticated,  exist  in 
a  social  context  and,  unless  designed  for  the  sake  of  design  itself,  serve  a  social  function"  (p.  2). 

Theorists  and  practitioners  in  the  field  of  instructional  development  (ID)  often  neglect  or  ignore  the 
social  context  into  which  their  instructional  products  are  intended  to  be  used.  Dalton  (1 989)  writes  about 
instructional  developers  that  "although  we  can  fill  instructional  gaps  with  fervor,  we  never  seem  to  examine 
our  solutions  in  light  of  the  wants  of  the  implementors"  (p.  22).  Burkman  (1987)  writes  that  instructional 
developers  commonly  believe  products  which  result  in  more  effective  and  efficient  instruction  will,  as  a 
direct  result,  be  attractive  to  potential  adopters.  Ralph  W.  Tyler  (1980)  adds  that  "many  developers  of 
technology  accept  the  view  that  as  time  passes,  there  will  be  increasing  use  of  the  innovation  until  it  has 
become  a  common  element  in  school  practice"  (p.  1 1).  The  basic  fallacy  pointed  out  by  these  statements  is 
that  development  of  eff  ;ctive  instruction  combined  with  the  passage  of  time  does  not  automatically  lead  to 
the  widespread  adoption  of  an  instructional  product.  Instructional  products  are  too  often  "designed  for  the 
sake  of  design  itself."  They  are  often  designed  without  regard  to  the  social  factors  that  influence  adoption 
and  utilization  and  this  results  in  the  development  of  instructional  products  that  are  technically  sophisticated 
and  instructionally  sound  but  that  nobody  uses. 

The  purpose  of  this  paper  is  to  discuss  the  process  by  which  instructional  products  are  developed 
and  to  describe  two  emerging  theories  that  might  link  the  people  who  design  instructional  technology  more 
closely  to  the  people  who  are  the  intended  users  of  the  technology.  The  theories  of  User-Oriented 
Instructional  Development  (UOID)  (Burkman,  1987)  and  Adoption  Analysis  (Farquhar  &  Surry,  1994)  are 
potentially  powerful  tools  for  instructional  developers.  By  incorporating  these  two  theories  into  their 
design  activities,  instructional  developers  can  create  products  that  are  not  only  "effective  and  efficient"  but 
that  are  also  useful  and  desirable  to  the  people  who  are  the  intended  users. 

Limitations  of  Existing  Product  Development  Models 

If  we  accept  the  premise  that  instructional  designers  often  neglect  or  ignore  the  social  context  into 
which  their  products  are  to  be  used,  the  obvious  next  step  to  ask  why  this  is  the  case.  One  likely  reason  for 
this  neglect  can  be  found  by  examining  the  theoretical  models  commonly  used  in  the  field  of  instructional 
technology.  These  models  are  used  by  instructional  designers  and  systems  developers  to  manage  and 
organize  instructional  development  activities  and  to  communicate  the  overall  process  to  clients 
(Gustafson,1991).  Instructional  development  models  provide  the  procedural  framework  by  which 
instructional  products  are  produced. 

There  are  numerous  models  of  instructional  development.  Gustafson  (1991 )  skillfully  organizes 
many  of  the  most  widely-used  instructional  development  models  into  a  logically  organized  taxonomy. 
Gustafson  classifies  the  models  into  Classroom  ID  Models,  Product  Development  Models,  and  Systems 
Development  Models.  For  the  purpose  of  this  paper,  we  will  primarily  discuss  the  product  development 
models. 

Perhaps  the  most  widely  used  instructional  development  model  is  the  Dick  and  Carey  Model 
(1990).  While  Gustafson  classifies  this  as  a  Systems  Development  Model,  it  is  also  commonly  used  by 
instructional  product  developers.  The  Dick  and  Carey  Model  (see  Figure  1 )  describes  a  development  process 
that  begins  with  the  identification  of  goals  and  proceeds  through  formative  evaluation,  revision  and 
summative  evaluation.  There  is  little  doubt  that  the  model  provides  a  valuable  description  of  all  of  the  key 
ID  activities  and  places  them  in  a  logical  sequence.  Notably  lacking  from  this  model,  however,  is  any 
mention  of  the  social  context  in  which  the  product  will  be  implemented. 
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Figure  1 .  The  Dick  and  Carey  Model  of  Instructional  Development 

As  with  the  Dick  and  Carey  Model,  other  widely  used  product  development  models  also  fail  to 
account  for  social  context.  Gustafson  (1991)  writes  that  the  goal  of  product  development  models  is  "to 
prepare  an  effective  and  efficient  product  as  quickly  as  possible"  (p.7).  While  all  three  of  the  product 
development  models  reviewed  by  Gustafson  describe  a  logical  process  for  developing  "an  effective  and 
efficient  product",  none  of  them  contains  a  thorough  discussion  of  the  need  to  analyze  the  social  context  in 
which  the  product  will  be  used.  In  fact,  only  one  of  the  three,  The  Van  Patten  Model  (1989),  even 
mentions  the  implementation  or  continuing  maintenance  of  an  instructional  product. 

In  reviewing  Systems  Development  Models,  Gustafson  writes  that  such  models  usually  call  for  an 
extensive  analysis  of  the  use  environment  before  instructional  development  even  begins.  Of  the  five 
systems  models  reviewed  by  Gustafson,  two  --  The  IDI  Model  and  The  Diamond  Model  -  do  discuss  in 
some  detail  the  need  .'or  an  analysis  of  the  social  context.  The  IDI  Model  (Twelker,  1972)  calls  for  an 
analysis  of  the  audience,  organizational  personnel,  and  organizational  resources  before  development  begins. 
The  Diamond  Model  (1989)  goes  even  further  than  the  IDI  Model  and  calls  for  an  analysis  of  societal  and 
organizational  needs  and  for  an  examination  of  human  and  organizational  resources  before  development. 

The  examination  of  the  preceding  instructional  development  models  leads  to  three  important 
conclusions.  First,  none  of  the  most  widely  used  product  development  models  include  an  analysis  of  the 
social  context  as  an  important  part  of  the  development  process.  Second,  product  development  models  do 
not  always  mention  adoption  and  diffusion,  and  when  they  do,  adoption  and  diffusion  are  typically 
considered  near  the  end  of  the  development  process,  usually  after  the  product  has  been  developed.  Third, 
while  some  systems  development  models  do  tend  to  call  for  a  thorough  analysis  of  social  context,  these 
models  are  not  often  used  to  guide  the  production  of  specific  instructional  products  but,  rather,  are  reserved 
primarily  for  the  development  or  repair  of  broader  instructional  systems. 

User-Oriented  Instructional  Development  Theory 

As  we  have  seen  in  the  previous  section,  few  product  development  models  discuss  sufficiently  the 
need  to  examine  the  social  and  physical  environment  into  which  an  instructional  product  will  be  introduced. 
Ernest  Burkman  (1987)  was  one  of  the  first  writers  to  propose  that,  because  traditional  ID  models  fail  to 
adequately  account  for  the  social  context,  instructional  products  have,  as  a  direct  consequence,  failed  to  be 
widely  implemented.  Burkman  writes  that  instructional  technology  makes  extensive  use  of  the  research- 
development-diffusion  (RDD)  paradigm  and  that  many  such  endeavors  suffer  from  a  lack  of  utilization.  He 
adds  that  instructional  technology  has  experienced  a  lack  of  utilization  in  all  fields,  including  primary  and 
secondary  schools,  colleges  and  universities,  and  even  in  industry  and  the  military. 

In  order  to  correct  the  inadequacy  of  traditional  models,  and  to  increase  the  utilization  of  ID 
products,  Burkman  proposes  the  User-Oriented  Instructional  Development  Process  (UOID).  The  five  step 
process  (see  Figure  2)  calls  for  instructional  developers  to  identify  the  people  who  will  be  using  the 
product,  analyze  their  perceptions  about  the  product  and,  based  upon  that  analysis,  to  develop  products  that 
are  user-friendly. 
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5: 

Provide  post  adoption  support 

Figure  2.  The  Five  Steps  in  Burkman's  (1987)  UOID  Process 

Burkman  makes  an  interesting  and  valuable  point  in  describing  the  UOID  that  is  particularly 
germane  to  the  present  paper.  He  includes  three  major  differences  between  traditional  ID  models  and  his 
UOID  theory.  The  three  differences  serve  to  succinctly  define  the  differences  between  traditional  ID  theon 
and  the  emerging  theories  discussed  in  the  present  paper.  The  three  major  differences  between  emerging 
theories  and  traditional  models  as  identified  by  Burkman  are: 

I )         Traditional  models  do  not  call  for  instructional  developers  to  measure  the  perceptions  of 
potential  adopters  or  to  develop  perception-friendly  products 

2!         Traditional  models  do  not  ask  developers  to  formulate  messages  and  select 
communication  channels  in  order  to  create  favorable  perceptions. 

?)         Traditional  models  do  not  demand  that  instructional  developers  use  adoption  and 
implementation  success  as  criteria  for  evaluating  their  products. 

One  of  the  main  theoretical  foundations  that  underlies  Burkman's  UOID  Theory  is  H.  VI.  Rogers' 
( 1987)  theory  of  perceived  attributes.   Rogers  writes  that  all  innovations  can  be  thought  of  as  having  five 
general  attributes:  compatibility,  complexity,  observability,  relative  advantage  and  trialability.  The  theory 
states  that  potential  adopters  form  their  opinions  of  an  innovation  based  upon  their  perceptions  of  the 
innovation's  five  general  attributes.  Simply  put,  Rogers  theory  states  that  potential  adopters  arc  more  likely 
to  use  a  product  if  it  is  compatible  with  their  needs,  is  not  too  complex,  offers  observable  benefits,  provides 
some  advantage  relative  to  other  products,  and  can  be  tested  or  tried  out  prior  to  adoption.  Burkman  expands 
upon  Rogers  theory  and  applies  it  to  instructional  innovations  by  theorizing  that  instructional  developers 
can  increase  the  utilization  of  their  products  by  determining  and  accounting  for  the  perceptions  of  potential 
users. 

While  there  arc  no  published  studies  that  specifically  support  Burkman's  UOID  Theory,  there  is  a 
great  deal  of  research  to  support  the  theory's  underlying  foundation  that  perceived  attributes  play  an 
important  role  in  adoption.  Hurt  and  Hibbard  (1989)  write  "it  is  well-documented  that  the  characteristics  of 
an  innovation  as  perceived  by  potential  adopters  play  a  critical  role  on  the  rate  of  acceleration  of  the 
adoption  curve"  (p.  214).  Among  the  more  recent  studies,  Holloway  (1977)  found  that  perceptions  of 
relative  advantage  and  compatibility  were  influential  in  the  adoption  of  an  educational  innovation  by  high 
school  principals.  Moallcmian  (1984)  and  Weinstein  (1986)  also  found  that  perceptions  played  an 
important  role  in  adoption  in  educational  settings.  Surry  (1993)  found  that  the  perceptions  of 
compatibility,  complexity,  and  relative  advantage  were  important  considerations  in  the  adoption  of 
computer-based  instructional  modules  by  weather  forecasters.  Many  other  studies  (e.g.,  Sekhon.  1968; 
Rogers,  Daley,  &  Wu,  1982)  have  found  that  perceptions  played  a  significant  role  in  the  adoption  of 
innovations  outside  of  the  educational  field. 

Adoption  Analysis 

Adoption  Analysis  (Farquhar  &  Surry,  1994)  is  another  emerging  theory  that  is  based  upon  the 
assumption  that  existing  ID  models  are  inadequate  because  they  ignore  the  social  context  into  which  the 
instructional  product  will  be  introduced.  Adoption  Analysis  is  a  process  that  calls  for  a  thorough 
examination  of  both  the  context  in  which  an  instructional  product  will  be  used  and  of  the  people  who  will 
use  the  product.  Segal  (1994)  writes  that  "if,  as  in  the  significant  case  of  the  auto,  modern  technology 
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sol  ved  a  number  of  problems,  social  as  well  as  technical,  from  the  outset  it  simultaneously  bred  or  helped 
to  breed  several  others,  social  and  technical  alike"  (p.  30).    Adoption  analysis,  therefore,  can  be  defined  as 
a  process  that  seeks  to  determine  and  account  for  the  social  and  technical  problems  that  will  be  bred  by  the 
introduction  of  an  instructional  product  into  an  organization. 


Input/Output 


X  / 


Analysis  Tool 


Needs 
Assessment 


Front-end 
Analysis 


Input/Output 


Analysis  Tool 


C Hierarchy  of 
Objectives 


Figure  3.  Inputs  and  Outputs  of  Common  Analysis  Activities. 

An  analysis  phase  is  common  to  most  instructional  development  models.  Okey  (1990)  describes 
five  analysis  tools  that  are  commonly  used  by  instructional  developers.  As  shown  in  Figure  3,  the  five 
tools  described  by  Okey  can  be  applied  in  a  orderly  sequence  with  the  output  of  one  analysis  tool  providing 
the  input  for  the  next.  The  final  output  of  this  traditional  analysis  sequence  is  a  hierarchical  organization  of 
instructional  objectives.  This  hierarchy  of  objectives  is  then  used  as  a  framework  by  instructional 
developers  when  developing  instructional  strategies  and  creating  support  materials. 

The  analysis  tools  described  by  Okey  are  very  valuable  in  the  development  of  instructional 
objectives  but  do  not  inform  the  developers  about  the  social  context  in  which  the  instructional  product  will 
be  used.  Adoption  analysis  can  be  used  as  an  additional  analysis  tool  for  instructional  developei  "  who  are 
concerned  with  the  adoption  and  implementation  of  their  products.  The  focus  of  adoption  analysis  is  on 
the  individual  and  organizational  factors  that  could  impede  or  facilitate  the  adoption  and  integration  of  a  new 
technology. 
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Figure  4.  Factors  Affecting  The  Adoption  Of  An  Instructional  Product 

The  characteristics  of  the  individuals  who  will  ultimately  use  an  instructional  product  can  play  an 
important  role  in  whether  or  not  the  product  is  adopted.  Farquhar  and  Surry  (1994)  define  individual 
characteristics  as  "all  of  the  skills,  attitudes,  perceptions,  and  knowledge  possessed  by  the  people  who  will 
use  the  technology."  As  shown  in  Figure  4,  Individual  Characteristics  can  be  divided  into  the  characteristics 
of  the  user  population  and  the  perceptions  that  the  users  have  of  the  instructional  product.  This  focus  on 
perceptions  is  closely  related  to  Burkman's  User-Oriented  Instructional  Development  Process. 

In  addition  to  individual  factors,  the  characteristics  of  the  organization  into  which  an  instructional 
product  will  be  used  often  determine  whether  or  not  the  product  is  adopted.  Organizational  factors  (see 
Figure  4)  include  all  of  the  personnel,  expertise,  attitudes,  hardware,  software,  facilities,  and  services 
available  within,  or  to,  an  organization.  Organizational  factors  obviously  play  a  major  role  in  the  initial 
adoption  of  an  innovation  but  'heir  most  important  role  is  likely  in  facilitating  or  hampering  the  continued 
use  of  an  instructional  product  in  the  years  after  its  initial  adoption. 

In  conducting  an  adoption  analysis,  the  instructional  developer  should  seek  to  answer  a  series 
questions  concerning  the  individual  and  organizational  factors  that  might  affect  adoption  of  the  product. 
Figure  5  lists  several  of  the  most  important  questions  that  can  help  to  inform  an  adoption  analysis. 


Individual  Factors 

User  Characteristics 

What  motivation  do  the  users  have  for  using  this  product? 

Do  the  users  have  the  necessary  technical  skills  to  use  this  product? 

Do  the  users  have  special  needs  that  this  product  should  address? 

Have  the  users  used  technology  of  this  nature  in  the  past? 

Do  the  users  have  the  knowledge  base  to  use  this  product? 

Perceived  Attributes 

Is  this  product  compatible  with  the  needs  and  culture  of  the  users? 
Is  this  product  compatible  with  the  users  work  or  study  schedule? 
Is  this  product  compatible  with  existing  hardware  and  software? 
Is  the  product  perceived  as  too  complex  or  too  simple? 

Is  the  product  perceived  as  offering  any  advantages  over  previous  or  competing  methods  of 
instruction? 

Do  the  users  pensive  that  they  will  experience  observable  benefits  from  using  this  product? 
Do  the  users  peice  e  that  they  will  be  able  to  test  or  trial  the  product  prior  to  full 
implementation? 
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Organizational  Factors 


Physical  Environment 

Where  will  the  product  be  used? 

How  will  the  product  be  used  in  the  daily  operations  of  the  organization? 
Does  the  organization  possess  the  hardware  needed  to  use  the  product? 
Does  the  organization  possess  the  necessary  labs  and  classroom  spaces? 
Who  will  install  the  product  and  any  other  complementary  equipment? 

Support  Environment 

Who  will  deliver,  maintain,  and  administer  the  product? 
Who  will  organize  and  deliver  any  needed  follow-up  training? 
How  will  supplies  be  stored,  ordered,  paid  for,  and  delivered? 
Do  key  decision  makers  support  or  oppose  the  product? 

Figure  5.  Key  Questions  That  Inform  Adoption  Analysis 

While  organizational  and  individual  factors  are  the  input  needed  to  conduct  an  adoption  analysis,  an 
adoption  plan  is  the  output  (see  Figure  6).  The  adoption  plan  should  specify  the  factors  that  will  most 
likely  facilitate  or  hamper  the  adoption  and  continued  use  of  an  instructional  product.  The  plan  should 
include  specific  design  features  that  will  make  the  instructional  product  more  likely  to  be  adopted  and 
maintained.  A  simplified  example  is  that  if  the  adoption  analysis  determines  that  none  of  the  workstations 
in  an  organization's  training  lab  have  CD  ROM  drives,  then  the  instructional  product  should  not  be 
packaged  in  or  require  the  use  of  CD  ROMs. 


Figure  6.  The  Input  and  Output  of  an  Adoption  Analysis. 


Recommendations 

The  theories  of  user  oriented  instructional  development  and  adoption  analysis  have  the  potential  to 
be  powerful  tools  for  instructional  designers.  It  is  possible  that  by  incorporating  these  theories  into  the 
development  process,  instructional  designers  can  develop  products  that  are  both  instructionally  effective  and 
desirable  to  the  people  who  are  the  intended  users  of  the  product.  The  following  recommendations  are 
provided  in  the  hope  that  they  will  be  incorporated  into  future  ID  projects: 

/)         Designers  and  developers  of  instructional  technology  should  consider  the  adoption  of  their 
products  as  carefully  as  they  consider  the  instructional  effectiveness  of  their  products. 

Developing  effective  and  efficient  instructional  products  does  not  necessarily  mean  that  the 
products  are  desirable  or  useful  to  potential  adopters.  The  field  of  instructional  development  has  made  great 
breakthroughs  in  designing  and  developing  effective  instruction.  Few  breakthroughs  have  been  made, 
however,  in  developing  products  that  people  want  to  use.  One  of  the  basic  tenets  of  instructional 
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therefore,  that  when  an  instructional  product  is  not  adopted,  instructional  developers  often  blame  the 
potential  adopters.  Another  basic  tenet  of  the  f,eld  should  be  "if  the  product  was  not  adopted,  it  means  the 
design  of  the  product  did  not  adequately  plan  for  adoption." 

2)  Designers  and  developers  of  instructional  technology  should  incorporate  the  theories  of  user- 
oriented  instructional  development  and  adoption  analysis  into  their  instructional  development  models.. 

If  instructional  developers  are  to  consider  adoption  of  their  products  as  carefully  as  they  consider 
instructional  effectiveness,  then  current  models  of  instructional  development  will  be  insufficient  for  the 
task.  Emerging  theories  that  place  an  emphasis  on  the  user  and  on  the  social  context  in  which  a  product 
will  be  used  can  be  incorporated  into  existing  product  development  models. 

3)  Research  should  be  undertaken  to  determine  if  incorporating  user-centered  instructional  design  and 
adoption  analysis  into  instructional  development  models  results  in  increased  adoption  and  utilization. 

The  theories  of  user-oriented  instructional  development  and  adoption  analysis  have  not  been 
examined  in  practical  settings.  There  is  no  published  evidence  to  suggest  that  employing  either  of  the 
theories  will  result  in  the  increased  adoption  or  facilitated  implementation  of  an  instructional  product. 
Research  into  the  effects  of  these  theories  is  non-existent  and  urgently  needed.  Large-scale  longitudinal 
studies  that  examine  the  impact  and  effectiveness  of  these  theories  on  the  adoption  and  continuation  of 
instructional  products  would  be  very  useful.  However,  since  it  is  unlikely  that  either  of  these  emerging 
theories  is  actually  being  used  in  large-scale  development  projects,  case  studies  of  smaller  projects  are  a 
more  likely  research  design. 

Conclusion 

It  is  not  the  intention  of  this  paper  to  put  forth  a  new  model  of  instructional  development.  We 
agree  with  Gustafson's  (1991)  conclusions  that  "the  literature  is  replete  with  models,  each  claiming  to  be 
unique  and  deserving  of  attention"  (p.47)  and  "it  appears  that  well  over  half  of  the  ID  models  have  never 
actually  applied,  never  mind  rigorously  evaluated"  (p.  47).  The  last  thing  the  ID  field  needs  is  another 
untested  design  model  claiming  to  be  unique  and  valuable. 
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Figure  7.  Incorporating  adoption  analysis  into  the  Dick  &  Carey  Model. 

Instead  of  creating  a  new  model,  adoption  analysis  and  a  user-centered  focus  can  be  and  should  be 
incorporated  into  existing  instructional  development  models.  Figure  7  below  shows  how  the  theory  of 
adoption  analysis  can  be  incorporated  into  the  widely-used  Dick  &  Carey  Model.  As  shown  in  Figure  7, 
adoption  analysis  can  easily  incorporated  into  popular  exiting  ID  models  without  radically  altering  them  or 
detracting  from  their  instructional  effectiveness.  As  an  example,  in  the  Dick  &  Carey  Model,  after  the 
goals  of  the  project  are  determined,  the  project  team  could  conduct  an  adoption  analysis.  The  output  of  the 
analysis  (an  adoption  plan)  would  specify  the  individual  and  organizational  factors  that  might  facilitate  or 
impede  the  introduction  of  the  product.  This  information  could  be  used  to  modify  the  goals  of  the  project 
or  to  inform  key  design  decisions  during  each  stage  of  the  process. 
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Much  more  importantly  than  putting  forth  a  new  ID  model,  what  is  really  needed  is  a  new  way  of 
thinking.  Instructional  developers  should  consider  the  potentiai  adoption  and  implementation  of  their 
products  as  carefully  as  they  consider  the  instructional  outcomes.  Put  another  way,  the  value  of  an 
instructional  product  should  be  measured  by  the  degree  of  adoption  and  the  success  of  implementation  just 
as  much  as  it  is  now  measured  by  cognitive  and  affective  outcomes.  In  order  for  this  to  happen, 
instructional  developers  will  have  to  analyze  and  account  for  the  social  context  in  which  their  products  will 
be  used.  Also,  developers  will  have  to  make  adoption  and  dissemination  important  considerations  of  their 
design  models  throughout  the  entire  ID  process.  Adoption  analysis  and  user-oriented  instructional 
development  can  be  potentially  valuable  tools  for  instructional  developers  who  agree  that  this  new  way  of 
thinking  is  necessary. 
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Abstract 

Summary  writing  is  considered  to  be  one  of  the  power  generative  learning  strategies.  However,  there  is 
insufficient  research  to  guide  the  design  of  summary  writing  activities  in  computer-based  instruction. 
Examination  of  summary  research  suggests  that  variables  such  as  learner  characteristics,  task  condition  and 
presentation  features  influence  the  effectiveness  of  summary  writing  in  promoting  meaningful  learning. 
The  paper  attempts  to  identify  research  issues  as  well  as  design  guidelines  for  embedding  summary  writing 
activities  in  computer-based  instruction  in  these  three  areas. 


How  do  instructional  activities  influence  learning  outcomes?  According  to  cognitive  theorists, 
there  is  a  clear  distinction  between  the  role  of  the  teacher  and  the  role  of  the  student  in  the  learning  process 
(Shuell,1988).  The  teacher's  primary  responsibility  is  to  structure  the  learning  activities  in  such  a  way  that 
the  students  engage  in  the  necessary  information  processing  functions,  and  the  student's  responsibility  is  to 
carry  out  the  functions.  That  is,  instruction  influences  how  students  receive,  process,  and  store 
information;  it  is  then  these  student  thought  processes  that  directly  mediate  learning  and  achievement 
(Wittrock,  1986).  For  learning  to  occur,  the  critical  functions  of  receiving,  encoding,  and  retrieving  must 
be  performed  by  the  student.  Thus,  researchers  interested  in  improving  the  quality  of  instruction,  including 
computer-based  instruction,  must  focus  on  ways  to  help  students  engage  in  the  critical  thought  processes 
that  lead  to  effective  learning.  This  paper  will  highlight  research  related  to  various  issues  of  using 
summary  writing  as  a  learning  strategy.  The  paper  will  also  note  potential  research  questions  as  well  as 
instructional  design  recommendations  based  on  the  existing  research. 

Summary  Writing 

Summary  writing  is  considered  to  be  one  of  the  power  generative  learning  strategies  (Pressley, 
Johnson,  Symons,  McGoldrick,  &  Kurita,  1989).  Summaries  are  defined  as  condensed  statements  that 
represent  the  gist  or  essence  of  the  information  available  (Johnson,  1983).  Researchers  propose  a  variety  of 
descriptions  of  the  processes  involved  in  summary  writing.  Most  of  these  descriptions  include  at  least  the 
operations  of  selection  and  reduction  (Brown  &Day,1983;  Hidi  &  Anderson,  1986).  Johnson  (1983),  for 
example,  suggests  that  there  are  six  components  to  the  generation  of  a  summary:  (a)  comprehending  the 
individual  propositions,  (b)  establishing  connections  between  propositions,  (c)  identifying  the  constituent 
structure,  (d)  remembering  the  information,  (e)  selecting  the  information  to  be  presented  in  the  summary, 
and  (f)  formulating  a  concise  and  coherent  verbal  representation  of  that  information. 

Many  researchers  argue  that  the  ability  to  summarize  is  important  for  understanding  and 
remembering  text.  Kintsch  and  van  Dijk  (1978)  suggest  that  readers  form  a  gist  which  represents  their 
overall  comprehension  of  the  text  while  reading  text.  According  to  these  authors,  the  reading  process 
involves  multiple,  cyclical  processing  of  propositions  to  arrive  at  macrostructures  of  the  discourse  using 
macrorules  and  generation  of  a  gist  of  the  discourse  based  on  the  macropropositions  in  memory,  control 
schema,  and  general  knowledge.  Under  this  conception,  summarizing  becomes  an  integral  component  of 
comprehension. 

Summary  writing  has  traditionally  been  used  as  a  way  to  assess  recall  and  comprehension  (Head, 
Readence,  &  Buss,  1989).  More  recently,  however,  researchers  have  proposed  that  the  processes  involved  in 
summarizing  can  facilitate  learning  since  they  help  learners  clarify  the  meaning  and  significance  of  the  new 
information  (Brown,  Campione,  &  Day,  1981;  Hidi  &  Anderson,  1986).  This  new  look  at  summary 
writing  is  consistent  with  the  generative  learning  view  of  summary  writing  as  an  effective  method  for 
constructing  meaningful  relationships  during  learning  (Wittrock,  1990).  Whether  summarizing  is  truly  an 
automatic  by-product  of  reading  or  not,  summary  writing  as  a  generative  activity  makes  these  mental 
processes  conscious.  This  mindful  activity  makes  it  possible  for  teachers  to  assess  students'  comprehension 
of  the  text,  for  students  to  monitor  their  own  comprehension,  and  for  students'  learning  to  become  more 
meaningful  and  memorable. 

Effects  of  Summary  Writing  on  Learning 
Although  there  are  many  empirical  studies  of  summary  writing,  a  majority  of  them  focus  on 

effects  of  various  treatments  on  students'  summary  writing  ability.  Relatively  few  studies  are  available  to 

examine  the  effects  of  summarizing  on  learning. 

Wittrock  and  Alessandrini  (1990)  studied  the  generation  of  summaries  and  analogies  in  light  of 

students'  analytical  and  holistic  abilities.  Fifty-seven  undergraduate  students  were  randomly  assigned  one  of 
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three  treatments:  read  text,  generate  analogies,  and  generate  summaries.  All  students  read  a  50-paragraph 
passage.  For  each  of  the  fifty  paragraphs,  students  were  required  to  read  and  reread,  generate  an  analogy,  or 
generate  a  paraphrased  summary.  Analysis  of  student-generated  analogies  and  summaries  indicated  that  the 
two  generation  treatments  induced  the  desired  generative  processing,  and  that  summaries  were  somewhat 
easier  to  construct  than  analogies.  Results  on  the  posttest  revealed  that  students  in  summary  generation  and 
in  analogy  generation  groups  learned  significantly  more  than  did  students  in  the  read  only  group.  Analysis 
of  time  on  task  revealed  that  there  was  no  significant  difference  in  the  amount  of  time  spent  learning  the 
text. 

Ross  and  Di  Vesta  (1976)  studied  the  effects  of  oral  summary  activities  on  college  students' 
retention.  During  the  reading  phase,  162  female  undergraduate  students  independently  read  a  passage  and 
were  given  one  of  two  kinds  of  learning  instructions:  students  were  told  to  read  the  passage  or  they  were 
told  prepare  an  oral  summary  from  the  reading.  Pairs  of  students  were  then  randomly  assigned  to  one  of 
three  participation  activities:  present  oral  summary  to  partner,  listen  to  oral  summary  of  partner,  or  engage 
in  neither  activity  during  review.  All  students  completed  a  multiple  choice  test  and  a  delayed  short-answer 
test .  On  measures  of  retention,  results  showed  that  the  multiple-choice  test  scores  for  students  who 
presented  oral  summaries  and  students  who  listened  to  oral  summaries  were  significantly  higher  than  those 
of  students  in  the  control  condition.  On  the  short-answer  test,  students  who  presented  oral  summaries 
performed  significantly  better  than  students  who  listened  to  summaries,  who  in  turn  performed  significantly 
better  than  students  in  the  control  condition.  Further  analysis  showed  that  the  amount  of  time  spent 
summarizing  was  highly  correlated  with  summarizers'  performance  on  both  multiple  choice  and  short 
answer  measures.  Overall,  the  authors  concluded  that  subjects  demonstrated  a  25%  advantage  in  retention 
after  engaging  in  oral  summaries,  and  a  13%  advantage  in  retention  after  listening  to  partners'  summaries 
over  students  who  did  not  engage  in  any  summary  activity. 

Design  Considerations  of  Summary  Writing  Activities  in  CBI 
Viewing  summary  writing  as  a  generative  activity  requires  thai  certain  instructional  design 
decisions  be  made  regarding  its  effective  use.  In  particular,  learner  characteristics,  task  conditions,  and 
presentation  features  are  expected  to  influence  the  effectiveness  of  summary  writing  activity  in  promoting 
meaningful  learning. 
Learner  Variables 

Research  in  aptitude-treatment  interactions  suggests  that  there  are  many  learner  characteristics  that 
may  influence  the  effectiveness  of  instructional  strategies  (Carrier  &  Jonassen,  1988).  The  most  critical 
learner  attribute  in  relation  to  generative  learning  activities  is  the  learner's  ability  to  carry  out  the  mental 
processes  required  by  the  learning  activity.  Generative  learning  activities  are  more  beneficial  if  students  can 
engage  in  constructive  mental  processes  without  being  distracted  by  the  mechanics  of  generative  task. 
Students'  prior  knowledge,  age,  and  metacognitive  potency  are  three  learner  characteristics  that  are  closely 
related  to  their  ability  to  benefit  from  summarizing  (Jonassen,  1988). 

Prior  knowledge.  Prior  knowledge  is  perhaps  the  most  relevant  and  strongest  predicting  factor  in 
student  learning  (Kintsch,  1988;  Tobias,  1987).  Sufficient  prior  knowledge  helps  a  reader  arrive  at  the 
correct  understanding  of  a  text  and  reduces  incorrect  inferences.  The  perception  of  learning  as  a  process  of 
constructing  meaningful  relationships  necessarily  implies  that  learning  must  be  built  upon  learners' 
existing  knowledge.  Sufficient  prior  knowledge  is  necessary  for  basic  reading  comprehension,  for 
determining  the  relative  importance  of  ideas,  and  for  generating  effective  summaries. 

Recht  and  Leslie  (1988)  investigated  the  effects  of  prior  knowledge  on  short-term  verbal  and 
nonverbal  memory  and  on  long-term  retention.  Sixty-four  junior  high  school  students  were  classified  as 
high  or  low  reading  ability  and  as  possessing  high  or  low  level  of  prior  knowledge  about  baseball.  Each 
student  read  a  story  about  a  baseball  game,  used  wooden  figures  to  reenact  the  story,  and  verbally  described 
the  events.  The  student  was  then  asked  to  summarize  the  story  and  sort  random  sentences  into  categories  of 
high,  medium,  or  low  importance  to  the  story.  Results  on  the  reenactment  scores  showed  that  memories  of 
high'knowledge  students  were  significantly  better  than  those  of  low  knowledge  students.  Verbal  retelling 
scores  showed  that  high  knowledge  students  recalled  significantly  more  than  low  knowledge  students.  High 
knowledge  students  produced  summaries  that  contained  more  correct  information  from  the  story,  and  were 
more  like  expert  summaries  than  students  with  low  knowledge.  In  addition,  high  knowledge  students  were 
more  able  to  sort  sentences  into  categories  of  high,  medium,  and  low  importance  similar  to  those  of  experts 
than  were  low  knowledge  students.  These  results  were  similar  for  both  high  reading  ability  and  low  reading 
ability  students. 

In  designing  summarizing  activities  to  enhance  the  quality  of  instruction,  attention  must  be  given 
to  the'available  knowledge  and  experience  levels  of  the  learners.  It  is  perhaps  more  beneficial  to  embed 
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summary  activities  in  school  tasks  that  are  a  part  of  continuous  curriculum  units  since  these  curriculum 
activities  often  build  upon  prior  learning  of  the  same  content  area  than  in  artificial  learning  tasks. 

Age  and  metacognition.  Age  and  metacognitive  awareness  are  two  related  characteristics  that 
influence  students'  ability  to  carry  out  the  mental  processes  required  by  summary  writing  tasks.  As  students 
gain  more  experience  in  academic  tasks,  they  are  likely  to  gain  skills  in  carrying  out  ceitain  generative 
learning  tasks,  and  they  are  also  like  to  benefit  from  metacognitive  experiences. 

Garner  (1985)  investigated  older  students'  ability  to  summarize  text.  One  hundred  and  twenty 
ninth-grade,  eleventh-grade  and  college  students  were  asked  to  produce  an  optimal  and  a  poor  summary  for  a 
three-page  gem  »al  science  article.  Student  summaries  were  scored  on  the  number  of  important  ideas 
included  and  succinctness.  On  the  number  of  important  ideas  included  in  summaries,  results  showed  that 
high  school  and  college  students  were  aware  that  important  ideas  from  a  descriptive  passage  should  be 
included  in  a  short  summary  of  the  passage.  Seventy-three  percent  of  all  the  students  differentiated  optimal 
and  poor  summaries  by  including  more  judged-important  ideas  in  the  optimal  summaries,  although  college 
students  showed  higher  awareness  than  high  school  students.  College  students  were  more  capable  of 
actually  including  important  ideas  in  their  optimal  summaries  than  high  school  students.  The  author 
concludes  that  high  school  and  college  students  are  well  awareness  that  important  ideas  should  be  included 
in  summaries,  and  that  this  awareness  is  matched  by  an  ability  to  actually  include  the  important  ideas  in 
summaries  in  college  students  but  not  in  high  school  students.  On  measures  of  succinctness,  however, 
both  high  school  and  college  students  were  significantly  less  aware  and  less  capable. 

Brown,  Day,  and  Jones  (1983)  studied  younger  and  older  students'  use  of  planning  strategies  in 
generating  written  summaries.  Fifty-seven  students  in  fifth-grade,  seventh-grade,  eleventh-grade  and  college 
learned  two  folk  stories  to  criterion.  The  students  were  then  asked  to  summarize  one  of  the  stories  from 
memory  three  times:  unrestricted,  in  40  words,  and  in  20  words.  Student  summaries  were  rated  on  three 
measures:  number  of  important  ideas  included,  amount  of  verbatim  information,  and  evidence  of  planning. 
On  the  number  of  important  ideas  included  in  summaries,  seventh  and  eleventh-graders  and  college  students 
exhibited  similar  patterns  in  including  important  idea  units  in  their  summaries,  but  college  students  showed 
greater  efficiency  at  including  more  idea  units  in  fewer  words.  Analyses  of  the  unrestricted  summaries  and 
the  40-word  summaries  showed  that  seventh  and  eleventh  grade  students  and  college  students  did  not  differ  in 
including  important  ideas  in  their  summaries;  all  groups  outperformed  fifth-grade  students.  In  the  20-word 
summaries,  college  students  outperformed  fifth,  seventh,  and  eleventh  grade  students  in  including  important 
idea  units  in  their  summaries.  On  amount  of  verbatim  information,  fifth  and  seventh  graders  produced  more 
verbatim  summaries  while  the  majority  of  the  eleventh  graders  and  college  students  wrote  summaries  in 
their  own  words.  On  evidence  of  planning,  young  students  tended  to  run  out  of  space  and  could  not  include 
important  ideas  from  the  second  half  of  the  stories  due  to  lack  of  planning.  Overall,  significantly  more 
eleventh  graders  and  college  students  exhibited  spontaneous  planning  behaviors  than  did  fifth  and  seventh 
graders. 

It  seems  that  without  extensive  strategy  training,  summarizing  as  a  learning  mechanism  is  more 
beneficial  for  older  students  than  for  younger  students. 
Task  Conditions 

The  nature  of  the  summary  writing  condition  can  be  expected  to  mediate  the  quality  of  the 
summaries  produced  (Hidi  &  Anderson,  1986;  Johnson,  1983).  To  the  extent  that  these  conditions 
influence  the  cognitive  operations  of  the  summarizer,  they  will  influence  the  effects  on  learning  outcomes. 
Four  such  conditions  are  1)  the  availability  of  the  lesson  content  during  summarizing,  2)  the  frequency  of 
summary  activities  within  a  lesson,  3)  type  of  summary  required,  and  4)  the  presentation  features  of  the 
lesson. 

Availability  of  the  lesson  content  while  summarizing.  Hidi  and  Anderson  (1986)  propose  that  if 
students  are  able  to  scan  the  material  repeatedly  during  summarizing,  the  demands  on  the  short-term 
memory  are  reduced,  and  there  are  more  cognitive  resources  available  for  other  operations  such  as  making 
fine  discriminations  about  the  importance  of  ideas  in  the  text,  clarifying  inconsistencies,  and  chunking 
larger  text  units.  Therefore,  the  tasks  of  discrimination,  comparison,  inference,  structure  assembly,  and 
writing  can  be  performed  more  deliberately.  More  importantly,  the  summarizer  can  repeatedly  check  the 
production  of  the  summary  against  the  lesson  content  for  accuracy. 

On  the  other  hand,  when  students  must  summarize  from  memory,  these  processes  are  given 
different  priorities,  assuming  that  students  have  not  completely  memorized  the  content  first.  The  process  of 
scanning  is  limited  to  retrieving  available  memory  traces  of  the  text.  The  tasks  of  discrimination, 
comparison,  and  inferencing  are  also  more  restrained  to  more  important  information  because  readers  are 
more  likely  to  remember  important  information  than  trivia!  information  (Mandler  &  Johnson,  1977;  van 
den  Broek,  1988).  Similarly,  processes  of  reorganization  and  summary  generation  must  rely  heavily  on 
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idiosyncratic  schemata  rather  than  on  the  text  itself.  When  the  propositions  are  recalled  from  memory  to  be 
summarized,  they  are  more  likely  to  appear  in  forms  of  personally  relevant  relationships  than  verbatim 
information  from  the  original  text.  Students'  mental  operations  can  be  expected  to  draw  heavily  on  the 
relation  between  the  new  information  and  the  learner's  own  experience  and  knowledge,  and  can  be  expected 
to  strengthen  and  stabilize  those  relationships. 

Empirical  evidence  that  examines  the  variable  of  text  availability  is  limited.  Hidi  (reported  in  Hidi 
&  Anderson,  1986)  reports  ambiguous  results  comparing  summarizing  under  text-present  and  text-absent 
conditions  in  two  studies.  In  the  first  experiment,  students  were  divided  into  two  groups:  text-present 
(summarize  text  while  looking  at  text),  and  text-absent  (summarize  text  from  memory).  Results  showed 
that  although  the  summaries  produced  in  the  two  groups  were  similar,  the  text-absent  condition  led  to 
higher  recall  scores  than  the  text-present  condition  one  week  after  the  treatment.  As  expected,  student 
summaries  in  the  text-absent  group  were  more  likely  to  deviate  from  the  original  text  and  combine 
propositions.  However,  these  results  were  not  replicated  in  the  second  experiment. 

Stein  and  Kirby  (1992)  studied  the  effects  of  text  presence  on  students'  summarizing  and  recall. 
Sixth-grade  students  summarized  a  passage  under  one  of  two  conditions:  text  present  or  text  absent. 
Students  were  then  asked  to  provide  oral  free  recalls  of  the  passage.  Student  summaries  were  scored  on 
amount  of  information  included,  the  ratio  of  the  number  of  important  propositions  over  the  total  number  of 
propositions  in  the  summaries,  and  on  the  degree  of  non-verbatim  integration  of  the  ideas  in  the  summaries. 
Student  recalls  were  scored  for  amount  of  information  included.  Results  were  mixed.  Specifically,  the  text- 
absent  condition  resulted  in  more  integrative  summaries  for  more  able  readers,  but  not  for  less  able  readers. 
Also,  the  amount  of  information  included  in  summaries  had  a  strong  effect  on  recall,  above  reading  ability, 
for  the  text-absent  group.  The  text-present  condition  led  to  higher  recall  scores  than  did  the  text  absent 
condition,  regardless  of  reading  ability  levels.  Reading  ability  and  depth  of  integration  in  summaries,  not 
number  of  propositions  included  in  summaries,  predicted  recall  performance  in  the  text-present  condition. 

Tsai,  Simsek,  and  Johnson  (1993)  studied  the  effects  of  text  presence  on  students'  summary 
production  and  comprehension  during  a  computer-based  tutorial.  Fifty-two  high  school  students  enrolled  in 
a  math  course  studied  a  computer  tutorial  and  were  asked  to  produce  three  on-line  summaries  as  they 
progressed  in  the  lesson.  The  students  were  randomly  assigned  to  one  of  four  treatments:  a)  text-present 
underlining,  b)  text-present  no  underlining,  c)  texs-absent  underlining,  and  d)  text-absent  no  underlining.  In 
the  two  text-present  conditions,  students  were  allowed  to  review  the  lesson  segment  while  attempting  to 
summarize  it.  In  the  two  text-absent  conditions,  students  were  informed  that  once  they  began  writing 
summaries,  they  would  not  be  able  to  review  the  lesson  segment.  In  the  two  underlining  conditions, 
students  received  the  instruction  with  the  important  phrases  underlined.  In  the  two  no-underlining 
conditions,  important  lesson  information  was  not  underlined.  Student  summaries  were  analyzed  for  number 
of  words,  number  of  important  ideas,  and  number  of  extraneous  ideas.  Students  completed  an  attitude 
survey  and  a  short-answer  post-test.  Results  showed  that  underlining  important  information  for  the  students 
led  to  significantly  longer  study  time  and  significantly  higher  post-test  scores  than  the  no-underlining 
conditions.  However,  being  able  to  review  the  lesson  while  summarizing  it  did  not  help  students  produce 
significantly  better  summaries  and  did  not  lead  to  significantly  higher  post-test  scores  than  students  in  the 
text-absent  conditions. 

Tsai  (1994)  examined  the  effect  of  lesson  availability  in  a  similar  study.  High  school  students 
studying  complex  computer-based  biology  lessons  were  randomly  assigned  to  several  summary-reading  and 
summary-writing  treatments.  Students  who  were  assigned  to  "summary  writing  from  memory"  conditions 
studied  the  lessons  one  section  at  a  time,  and  when  they  indicated  that  they  were  ready  to  summarize  the 
section,  they  were  not  allowed  to  review  the  section's  content  while  they  wrote  the  summary  using  paper- 
pencil  work  sheets.  Students  who  were  assigned  to  "summary  writing  with  review"  also  studied  the  lessons 
one  section  at  a  time.  When  they  indicated  that  they  were  ready  to  summarize  each  section,  they  were  given 
the  choice  of  reviewing  the  content  while  writing  the  summary  using  paper  and  pencil  work  sheets.  The 
students'  summaries  were  scored  for  number  of  words  produced,  and  the  number  of  independent  clauses 
produced  that  matched  expert  summaries.  Students  also  completed  a  multiple-choice  test,  a  short-answer 
test,  and  an  attitude  survey.  Results  showed  that  students  who  were  giver,  access  to  the  lesson  while 
summarizing  it  produced  longer  summaries  and  included  more  ideas  that  matched  expert  summaries. 
Correlation  analysis  showed  that  summary  length  and  number  of  matching  ideas  in  the  summaries  were 
highly  correlated  to  students'  performance  on  the  achievement  tests.  However,  the  treatment  did  not  directly 
affect  achievement,  attitude  or  learning  time. 

Research  in  a  related  areas  such  as  adjunct  questioning  and  inference  verification  also  provides 
ambiguous  results  Hayes-Roth  and  Walker  (1979)  studied  the  effects  of  text  availability  on  students' 
ability  to  verify  inferences.  Eight  undergraduate  students  studied  four  pairs  of  text  passages  under  two 
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conditions.  In  the  text  condition,  students  read  the  passages  before  completing  an  inference  verification  test. 
They  were  encouraged  to  review  portions  of  the  texts  while  trying  to  judge  whether  the  inferences  were  true 
or  false.  In  the  memory  condition,  students  were  asked  to  read  the  passages  very  carefully,  and  were  asked 
to  complete  adjunct  questions  on  arbitrary  information  in  the  texts  to  maximize  learning  from  the  texts. 
They  were  then  asked  to  complete  the  inference  verification  test  without  referring  back  to  the  texts.  All 
students  were  asked  to  justify  their  reasoning  for  their  answers  while  completing  the  inference  test. 
Performance  on  the  inference  verification  test  showed  no  significant  difference  between  the  text  and  memory 
conditions.  However,  on  the  accuracy  of  students'  justification  of  true  inferences  showed  that  the  proportion 
of  correctly  justified  true  inferences  was  significantly  higher  in  the  memory  condition  than  in  the  text 
condition.  Further  analyses  of  students'  verbal  protocols  indicated  that  students  in  the  memory  condition 
were  more  able  to  identify  the  critical  fact  underlying  true  inferences  than  students  in  the  text  condition. 

In  conclusion,  there  is  sufficient  evidence  to  suggest  that  access  to  text  during  summarizing 
mediates  the  quality  of  the  summaries.  However,  there  is  limited  evidence  about  how  this  influences  the 
quality  of  learning.  Theoretical  speculation  and  the  limited  research  evidence  suggest  that  summarizing 
from  memory  may  facilitate  long-term  retention  of  the  lesson  content.  A  concern  not  addressed  in  the 
existing  research  is  how  students  actually  use  the  opportunity  to  review  text  as  they  summarize.  In  a 
computer-based  Seaming  environment,  this  information  can  be  tracked,  and  it  may  provide  additional  insight 
to  how  availability  of  lesson  content  can  mediate  summarizing  and  learning. 

Frequency  of  summarizing.  Frequency  of  summarizing  refers  to  the  number  of  iioncumulative 
summaries  required  for  a  given  amount  of  lesson  content.  It  is  related  to  the  amount  of  information  to  be 
summarized  in  each  summary.  Current  understanding  of  the  spatial  and  temporal  limitations  of  the  short- 
term  memory  suggests  that  longer  segments  are  more  difficult  to  summarize  than  shorter  ones.  Hidi  and 
Anderson  (1986)  suggest  that  shorter  segments  are  likely  to  contain  fewer  ideas  and  ideas  which  are  closely 
related,  and  place  relatively  less  processing  demands  on  students'  short-term  memory  during  summarizing 
than  long  segments.  Exactly  how  this  variable  affects  learning,  however,  is  unclear. 

Spurlin,  Dansereau,  ODonnell,  and  Brooks  (1988)  had  48  undergraduate  students  summarize  a 
passage.  Students  were  randomly  assigned  to  one  of  three  conditions:  produce  frequent  noncumulative 
summaries  at  equally  spaced  intervals  throughout  the  passage,  produce  infrequent  summaries,  or  simply  read 
the  passage.  All  students  completed  a  delayed  test  requiring  them  to  write  a  well-organized  essay  reflecting 
what  they  had  read  five  days  after  the  treatment.  The  essays  were  measured  on  amount  of  recall  and  overall 
organization.  Analyses  of  student  summaries  showed  that  students  in  the  frequent  summarizing  group 
wrote  more  complete  summaries  than  students  in  the  infrequent  summarizing  group.  However,  on  both 
amount  of  recall  and  overall  organization  of  the  post-reading  essay  test,  only  the  infrequent  summarizing 
group  significantly  outperformed  the  control  group. 

Germane  (1921)  studied  the  effects  of  written  paragraph  summaries  on  students'  learning  of  science 
texts.  Nine  hundred  sixth-,  seventh-,  and  eighth-grade  students  were  randomly  assigned  to  summarizing 
group  or  re-reading  group  while  studying  two  science  articles.  Students  in  the  summarizing  group  were 
first  taught  how  to  summarize  two  sample  paragraphs.  Then  for  each  article,  students  were  asked  to  read 
carefully  and  summarize  each  paragraph.  A  specific  time  restriction  was  enforced.  Students  in  the  re- 
reading group  were  asked  to  read  the  article  as  many  times  as  they  could  within  the  same  time  limit. 
Immediate  and  delayed  free  recall  tests  were  given.  Results  showed  that  the  re-reading  group  performed 
significantly  better  on  both  the  immediate  and  the  delayed  recall  tests.  These  results  held  true  for  questions 
that  addressed  main  ideas  and  minor  ideas  in  the  articles.  The  author  speculated  that  the  summarizing  group 
was  handicapped  by  the  method  of  summarizing  each  paragraph  since  the  students  seemed  to  have  a  large 
number  of  isolated  facts  but  failed  to  see  the  relationship  and  grasp  the  article  in  its  entirety. 

Tsai  (1994)  in  the  study  mentioned  earlier,  randomly  assigned  high  school  students  to  Frequent  or 
Infrequent  summary  activities  while  they  studied  complex  biology  lessons  using  computer-based  lessons. 
Students  who  were  assigned  to  the  Frequent  treatments  were  required  to  read  or  write  summaries  for  each 
content  topic  of  the  lessons.  These  students  wrote  a  total  of  12  summaries  while  studying  two  lessons. 
Students  who  were  assigned  to  the  Infrequent  treatments  were  required  to  read  or  write  summaries  for  each 
major  content  unit  of  the  lessons.  These  students  wrote  a  total  of  4  summaries  while  studying  the  two 
lessons.  Results  showed  that  students  who  participated  in  frequent  summary-writing  activities  wrote 
significantly  more  words  in  total  than  students  who  participated  in  infrequent  summary-writing  activities. 
Again,  summary  length  was  highly  correlated  with  performance  on  achievement  tests  even  though  the 
treatment  itself  did  not  directly  lead  to  significant  differences  in  achievement  or  time. 

There  are  three  reasons  that  summarizing  relatively  longer  lesson  segments  may  have  positive 
effects  on  learning  and  retention.  First,  students  who  summarize  infrequently  are  expected  to  extend  more 
recall  effort  than  do  students  in  frequent  summarizing  group,  and  the  increased  effort  may  lead  to  more 
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effective  delayed  recall.  Second,  students  are  more  likely  to  encode  and  retrieve  meanings  and  relationships 
than  isolated  facts.  When  students  summarize  large  chunks  of  content,  they  must  identify  the  relationships 
among  various  concepts  in  the  lesson  whereas  summarizing  short  segments  of  content  may  force  them  to 
focus  on  isolated  concepts.  Third,  the  mechanics  of  frequent  activities  may  distract  learners'  attention  from 
the  content  of  the  lesson.  When  students  are  only  required  to  summarize  infrequently,  they  are  more  likely 
to  adopt  text  studying,  not  summarizing,  as  the  primary  task. 

In  conclusion,  the  frequency  of  summarizing  within  a  lesson  is  expected  to  mediate  summary 
production  and  learning.  Limited  empirical  evidence  suggests  that  the  shorter  a  lesson  segment  the  easier  it 
is  to  summarize.  Huw«.ver,  there  is  also  reason  to  speculate  that  summarizing  longer  lesson  segments  may 
be  more  beneficial  for  learning.  Clearly  this  issue  needs  further  examination. 

Type  nf  summary  reuqired.  Researchers  have  utilized  a  variety  of  types  of  summaries  to  facilitate 
learning.  Taylor  and  Beach  (1984),  for  example,  taught  students  to  recognize  and  use  the  existing  textual 
structure  to  guide  their  summary  production.  Wittrock  and  Alessandrini  (1990)  and  Tsai  (1994)  required 
students  to  summarize  "in  their  own  words."  Brown,  Day,  and  Jones  (1983)  required  students  to  first  learn 
the  text  "to  criterion,"  and  then  to  produce  summaries  according  to  various  length  constraints.  In  the  Ross 
and  Di  Vesta  (!976)  study  and  the  Recht  and  Leslie  (1988)  study  mentioned  earlier,  students  were  required  to 
produce  oral  summaries  of  the  material. 

Hidi  and  Anderson  (1986)  proposed  that  there  is  a  difference  between  producing  a  "reader-based" 
summary  and  producing  a  "writer-based"  summary.  Reader-based  summaries  place  significant  emphasis  on 
proper  planning  (Brown,  Day,  &  Jones,  1983)  as  well  as  effective  grammar  and  prose  structure  skills.  The 
purpose  of  the  reader-based  summaries  is  to  present  an  effective  communication  of  the  orginal  material  to 
others.  Writer-based  summaries,  on  the  other  hand,  primarily  serves  the  producers  of  the  summaries  to  help 
them  condense  the  information  for  their  own  use.  Writer-based  summaries  place  much  less  emphasis  on  the 
structure  of  the  writing,  and  more  emphasis  on  the  usefulness  of  the  content  for  the  summary  writers.  Hidi 
and  Anderson  (1986)  thus  suggest  that  writer-based  summary  writing  is  more  efficient  as  a  learning  strategy 
than  reader-based  summary  writing.  Although  this  proposition  seems  logical,  there  is  little  empirical 
evidence  so  far  to  demonstrate  the  relative  learning  value  of  these  two  types  of  summaries. 

Presentation  Feature  Manipulations.  Presenting  stimuli  with  distinctive  features  has  long  been  a 
prescribed  strategy  for  instructional  designers  to  help  students  focus  attention  on  important  information 
(Gagne,  Briggs,  &  Wager,  1988;  Wager  &  Gagne,  1988).  This  prescription  is  driven  by  the  understanding 
that  students  have  limited  short-term  memory  capacity,  and  instructional  events  must  take  advantage  of 
selective  perception.  The  underlining  strategy  is  widely  used  to  highlight  information  for  students  who 
may  not  be  sensitive  to  its  importance.  It  is  an  effective  tool  to  cue  the  students'  attention  which  in  turn 
results  in  better  retention  of  that  information. 

Philosophically  agreeable  or  not,  successful  students  are  often  those  who  are  able  to  recognize  as 
important  the  ideas  that  their  teachers  also  consider  important.  Many  researchers  believe  that  students  fail 
to  grasp  the  important  information  while  studying.  This  is  often  cited  as  a  major  obstacle  to  successful  use 
of  effective  learning  strategies  (Anderson  &  Armbruster,  1984;  Winograd  &  Bridge,  1986).  It  may  be 
necessary  to  direct  students'  attention  to  important  information  so  that  they  benefit  from  generative  learning 
strategies  more  efficiently. 

In  the  Tsai,  Simsek  and  Johnson  study  cited  earlier  (1993),  students  who  studied  the  computer- 
based  tutorial  with  important  information  underlined  tended  to  include  more  important  ideas  than  students 
who  were  not  aided  by  the  underlining,  although  the  differences  were  not  significant.  However,  the 
underlining  treatments  led  to  significantly  higher  post-test  scores. 

Garner  and  McCaleb  (1985)  studied  the  effects  of  cueing,  organization,  and  reduction  constraints  on 
undergraduate  students'  summary  production.  The  cueing  variable  included  3  treatments:  a)  no  cueing,  b) 
explicit  topic  sentences,  and  c)  signal  words  such  as  "important"  embedded  in  the  topic  sentences.  Two 
levels  of  organization  were  used:  a)  all  important  ideas  were  presented  together,  and  b)  important  ideas  were 
distributed  among  the  paragraphs.  Two  levels  of  reduction  constraints  were  studied:  a)  three-sentence 
summary,  and  b)  seven-sentence  summary.  Students  read  a  seven -paragraph  science  text  in  one  of  the 
treatments  and  wrote  summaries.  Student  summaries  were  scored  for  number  of  words,  number  of 
important  ideas,  evidence  of  deviation  in  order  of  information,  and  evidence  of  integration.  Results 
suggested  that  cueing  had  a  significant  effect  on  summary  scores.  Specifically,  both  cueing  treatments  led 
to  significantly  higher  number  of  important  ideas  in  student  summaries  than  no  cueing,  with  no  significant 
difference  between  the  two  cueing  conditions.  Also,  both  cueing  treatments  produced  significantly  more 
integration  of  ideas  in  student  summaries  than  no  cueing,  again  with  no  significant  difference  between  the 
two  cueing  groups. 
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It  is  arguable  that  the  very  strategy  of  highlighting  important  information  may  interfere  with 
students'  construction  of  personally  relevant  relationships  out  of  the  instructional  information.  The 
processes  of  integrating  the  information  and  selecting  important  ideas  may  be  reduced  to  a  more  mechanical 
process  of  locating  cued  phrases  in  the  text.  Nevertheless,  the  results  hold  significant  promises  for  the 
instructional  designer.  It  seems  possible  that  simple  manipulations  of  text  features  can  significantly 
increase  the  effective  use  of  complex  learning  strategies  such  as  summarization.  The  reality  of  educational 
practices  is  that  students  are  often  evaluated  against  what  instructional  designers,  teachers,  and  other  experts 
believe  to  be  important.  Thus,  a  pragmatic  task  of  designing  instruction  is  to  engage  students  in  effective 
generative  activities  in  order  to  learn  what  is  considered  important.  To  this  end,  simple  attention  focusing 
strategies  such  as  underlining  important  phrases  may  enhance  the  learning  outcome  of  summary  writing 
activities. 

Instructional  Design  recommendations 
Several  instructional  design  guidelines  can  be  drawn  from  the  research  evidence  in  this  area.  These 
recommendations  focus  primarily  on  designing  the  summary  writing  activities  to  accommodate  learner 
characteristics  to  reduce  unnecessary  demands  on  the  learner  while  using  summarization  as  a  learning  tool. 

1)  Summary  writing  strategy  can  generally  be  reliably  used  with  students  who  are  older,  have 
more  sophisticated  metacognitive  control,  or  possess  high  levels  of  prior  knowledg. 

2)  The  teacher  or  instructional  designer  may  first  test  to  make  sure  that  students  are  able  to  perform 
the  mechanics  required  by  the  summary  writing  task  with  relative  ease. 

3)  Highlight  important  information  for  students  to  guide  their  summaries,  especially  for  complex 
content,  and  for  students  with  low  prior  knowledge  or  low  language  who  are  less  able  to  differentiate 
between  important  and  not-important  information 

4)  For  learners  with  limited  experience  writing  summaries,  provide  structure  in  the  summary 
writing  activities  to  make  this  a  productive  learning  tool.  Provide  structure  and  prompts  (such  as  "look  for 
the  topic  sentence,  look  for  the  support  idea,  do  not  include  repetitive  exar..ples...").  Encourage  students  to 
provide  oral  summaries  if  writing  is  a  significant  challenge  for  them. 

5)  Use  summary  writing  activities  for  curriculum  topics  that  build  upon  each  other  so  that  students 
can  draw  on  their  knowledge  of  previous  topics  to  write  effective  summaries. 

6)  Encourage  "writer-based  summaries"  to  reduce  the  demand  on  writing  skills,  and  to  promote 
summary-writing  as  a  learning  tool.  Encourage  students  to  make  their  summaries  personally  meaningful. 
As  Stein  and  Kirby  (1992)  found,  depth  of  integration  in  students'  summaries  may  be  an  important 
predictor  of  depth  of  learning. 

7)  Avoid  using  length  constraints.  Length  constaints  tend  to  require  careful  planning  and  editing 
which  may  distract  from  learning  of  the  content. 

8)  Allow  students  to  review  the  lesson  while  summarizing  it.  To  make  this  a  useful  option, 
encourage  students  to  check  the  accuracy  of  their  summaries  during  such  reviews.  Also,  help  students 
develop  the  strategy  of  monitoring  their  comprehension  through  such  reviews. 

9)  Allow  for  frequent  summary  writing  activities  (Tsai,  1994),  but  avoid  making  summary  writing 
the  predominant  task  which  may  distract  from  learning  (Germane,  1921). 

Further  ^Research  Issues 
Based  on  the  above  findings,  several  research  issues  are  worth  exploring. 

1 )  It  is  clear  that  learner  characteristics  have  a  strong  impact  on  the  effective  use  of  summary 
writing  strategy  in  learning.  It  would  be  helpful  for  the  instructional  designer  to  have  a  clearer  set  of 
algorithm  in  selecting  the  key  learner  characteristics  to  be  studied  and  designing  the  most  appropriate 
summary  activities.  Work  in  mathemagenic  and  generative  learning  strategies  by  researchers  such  as  Pat 
Smith  promises  to  provide  some  clarification  in  this  area. 

2)  What  is  the  best  way  to  predict  the  appropriate  frequency  of  summary  writing  activities?  It 
seems  that  learner  characteristics,  complexity  of  materials,  and  desired  learning  outcome  are  important 
considerations.  But  the  guidelines  for  making  such  determination  should  be  derived  from  empirical 
evidence. 

3)  The  variable  of  availability  of  lesson  during  summarization  needs  further  study.  There  are  at 
least  three  questions  regarding  this  variable.  First,  what  kind  of  summaries  tend  to  be  produced?  There  is 
contradicting  evidence  so  far  on  the  effects  of  this  variable  on  summary  production.  One  of  the  proposed 
outcome  of  summarizing  from  memory  is  that  students  integrate  more  information  into  their  personal 
schemata  than  if  if  they  simply  copy  the  information  from  the  original  lesson. 

Second  question:  what  is  the  impact  on  different  kinds  of  learning?  It  has  been  hypothsized  that 
students  who  summarize  from  memory  will  produce  more  integrated  summaries,  and  will  demonstrate  a 
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hiaher  level  of  increase  in  iheir  capability  to  analyze  and  synthesize  the  information  than  students  who 
scanned  the  lesson  frequently  during  summarization.  Clarification  from  empirical  evidence  in  this  area  will 
guide  the  instructional  designer  in  determining  the  appropriate  strategies. 

Third  question:  what  strategies  do  students  use  to  write  summaries?  Tsai  (1 994)  suggested 
summarizing  from  memory  is  an  unfamiliar  task  for  many  students,  and  that  they  must  learn  the  strategy  of 
studying  the  lesson  more  carefully  before  beginning  their  summary  writing.  In  addition,  Tsai  (1994) 
suggested  that  when  students  do  have  the  option  of  reviewing  the  lesson  while  summarizing  it,  it  is  unclear 
how  they  actual!"  use  that  option.  If  they  do  not  review  the  lesson  to  increase  the  accuracy  of  their 
summaries,  then'this  option  is  not  necessary.  Thus,  examination  of  student  behaviors  under  different 
conditions  of  this  variable  will  be  helpful  in  designing  the  appropriate  summary  activities. 

4)  The  effects  of  writer-based  and  reader-based  summaries  should  be  examined.  Midi  and  Anderson 
(1986)  would  propose  that  for  learning  purposes  writer-based  summaries  are  more  efficient  and  preferable  to 
reader-based  summaries.  It  could  be  argued,  however,  that  the  process  of  planning  and  structuring  a  "good" 
reader-based  summary  is  hichly  beneficial  to  learning,  especially  for  people  who  believe  that  "the  best  way 
to  learn  a  topic  is  to  teach  it""."  Thus  the  question  of  "is  writer-based  summary  writing  more  efficient  and 
beneficial"  should  be  studied.  Related  to  this  question  is  the  need  to  examine  whether  summary  length 
constraints  enhance  or  reduce  efficiency  of  learning  for  the  strategy  of  summary  writing. 

Conclusion 

Embedding  generative  learning  activities  in  CBI  courseware  has  been  recommended  for  promoting 
deep  processing  in  CBI  (Jonassen,  1988).  However,  it  is  important  that  instructional  design  decisions  for 
these  activities  are  informed  by  sound  theoretical  and  empirical  evidence.  This  paper  focused  on  one  of  the 
more  powerful  generative  learning  activities,  namely,  summary  writing.  It  reviewed  current  understanding 
of  the  mental  processes  involved  in  generating  summaries,  and  examined  instructional  design  variables  that 
are  believed  to  mediaie  summary  production  and  its  effects  on  learning. 

The  mental  processes  involved  in  producing  summaries  -  such  processes  as  scanning  information, 
comparing  information  for  importance,  and  reorganizing  the  important  information  into  a  coherent  format  - 
promote  deep  processing.  These  processes  also  provide  an  opportunity  to  strengthen  the  conceptual 
relationships  that  the  learner  may  have  subconsciously  constructed  during  learning.  There  is  convincing 
evidence  that  summary  writing  activities  can  facilitate  student  learning.  However,  many  instructional 
design  decisions  regarding  the  effective  use  of  summarizing  have  not  been  resolved.  Specifically,  limited 
evidence  is  available  to  determine  whether  it  is  more  beneficial  to  require  students  to  summarize  from 
memory  than  to  summarize  while  reviewing  the  lesson  content.  It  is  unclear  how  the  frequency  of 
summary  activities  within  a  given  lesson  affects  summary  production  and  its  effects  on  learning.  In 
addition  we  need  to  further  study  the  impact  of  highlighting  information  on  the  effectiveness  of  summary 
writing  activities.  Research  in  these  areas  can  provide  insights  into  the  mental  processes  of  summarizing  as 
well  as  useful  guidelines  for  instructional  designers  in  designing  effective  summarizing  activities. 
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Abstract 


An  experiment  is  reported  in  which  subjects  worked  with  an  interactive  videodisk  program  about  cheese 
making  on  the  basis  of  the  same  set  of  instructional  objectives  but  in  different  experimental  conditions  with 
respect  to  learner  control.  The  puipose  of  the  study  was  to  investigate  the  relationship  between  learner- 
controlled  or  program-controlled  length  of  video  segments  and  test  performance  on  post-tests  and  retention 
tests.  Also,  differences  between  self-chosen  and  program-controlled  segment  length  regarding  the 
investment  of  mental  effort  of  subjects  were  examined.  The  results  show  that  long  program-controlled  video 
segments  have  some  advantage  over  interactive  video  when  learning  factual  information.  However,  well- 
designed  interactive  video  applications  may  offer  a  comfortable  work  environment  that  allows  for  a  relaxed 
use  of  mental  effort.  The  results  of  the  experiment  seem  to  suggest  that  video  segments  with  a  length  of 
about  3  minutes,  containing  enough  factual  information  for  answering  8  or  9  questions,  may  appear  to  be 
optimal  building  blocks.  Subsequent  experiments  should  reveal  to  what  extent  this  suggestion  is  valid. 


Introduction 

Verhagen  (1992a,  1992b)  undertook  a  study  to  gather  insight  about  the  length  of  segments  in  interactive 
video  programs  on  the  basis  of  the  following  central  question: 

What  is  the  optimum  length  of  well-designed  audio-visual  segments  to  present  factual  information  via  an 
interactive  video  program  to  learners  who  possess  certain  characteristics? 

In  his  study,  five  experimental  conditions  were  used  in  an  attempt  to  answer  eleven  more  elaborate  research 
questions.  The  study  was  carried  out  with  a  specially  produced  videodisk  program  about  cheese  making. 
This  program  contains  252  information  elements  which  form  a  connected  discourse  of  36  minutes  if  the 
program  is  played  linearly  without  stopping.  An  information  element  is  defined  as  one  uninterrupted 
statement  of  the  off-screen  narrator  about  which  one  factual  question  can  be  put.  In  two  conditions, 
subjects  couid  choose  segment  length  for  themselves  by  interrupting  the  video  presentation  anytime  they 
wanted.  In  the  other  three  conditions,  fixed  segment  lengths  were  used  to  study  recall  differences  with  self- 
chosen  segment  length.  Subjects  were  235  first-year  students  of  several  technical  and  social  science 
departments  at  a  single  university  in  the  Netherlands. 

Verhagen's  experiment  was  essentially  a  memory  experiment  in  which  the  program  was  used  to  find  out 
what  segment  lengths  are  preferred  by  learners.  Each  subject  worked  individually  with  the  program. 

The  results  showed  self-chosen  segment  lengths  of  which  the  means  per  subject  varied  from  2.19  to  87.50 
information  elements  (which  is  in  time  a  range  from  about  16  seconds  to  1 1  minutes).  The  mean  of  mean 
length  was  12.70  information  elements  (which  equals  about  1.7  minutes)  with  a  standard  deviation  of  1 1.77 
information  elements.  Details  are  reported  by  Verhagen  (1992a,  1992b). 

The  longer  the  video  segments  were,  the  lower  was,  on  average,  the  recall  of  factual  information  when 
answering  questions  directly  after  watching  a  segment.  The  differences  between  self-chosen  segment  lengths 
and  fixed  segment  lengths  appeared  thereby  to  be  small  with  the  exception  of  the  so-called  Linear 
Condition.  The  Linear  Condition  is  a  fixed-length  condition  in  which  the  subjects  are  forced  to  watch  the 
whole  program  without  stopping.  In  this  condition,  the  direct-recall  performance  was  substantially  lower 
(54%  correct  answers)  than  in  any  of  the  other  conditions  (around  80%  correct  answers). 

After  three  weeks,  delayed  recall  was  measured  with  a  retention  test.  In  this  case,  subjects  of  the  Linear 
Condition  performed  equally  well  or  better  than  all  others  while  on  average  questions  about  long  fixed 
segments  were  answered  better  than  questions  about  short  fixed  segments.  The  questions  about  the  long 
fixed  segments  often  also  yielded  a  better  performance  than  the  retention  scores  of  subjects  who  worked  with 
self-chosen  long  segment  lengths.  Inspired  by  work  of  Salomon  (1984)  and  of  Cennamo,  Savenye,  & 
Smith  (1991),  Verhagen  (1993)  argues  that  the  success  of  the  longer  segments  may  be  attributed  to  larger 
amounts  of  invested  mental  effort  (AIME)  due  to  perceived  more  difficult  demand  characteristics  (PDC's)  of 
the  expected  longer  segments. 
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Rules  about  what  to  regard  as  'optimal'  when  deciding  on  segment  length  did  not  emerge.  The  findings 
gave,  nevertheless,  reasons  to  formulate  a  few  tentative  guidelines  f  or  the  design  of  segments.  The 
guidelines  that  are  relevant  for  the  present  study  are  as  follows: 

"If  factual  information  is  presented  to  university  freshmen  and  a  systematic  presentation  with  off- 
screen narration  is  used,  a  wide  variety  of  segment  lengths  is  feasible.  If  a  safe  range  is  sought  in  which 
initial  remembering  is  to  reach  at  least  70%,  segment  lengths  up  to  22  information  elements  are 
appropriate.  Using  the  condensed  presentation  rate  of  the  experimental  video  program  used  in  the  study,  up 
to  about  3  minutes  of  presentation  time  can  be  used."  (Verhagen,  1992b,  f.280) 

"For  optimal  remembering,  learners  should  be  strongly  urged  to  invest  substantial  mental  effort. 
This  deepens  the  initial  processing,  which  enhances  long-term  storage  of  the  information.  Choosing  long 
fixed  segments  helps,  but  if  the  information  load  goes  beyond  the  ability  of  the  learners  they  may  find  the 
task  unpleasant.  On  the  other  hand,  segments  that  are  too  short  may  be  so  easy  that  learners  are  not 
compelled  to  invest  effort,  and  thus  hamper  adequate  storage  into  memory."  (Verhagen,  1992b,  p.260-261) 


Need  for  further  study 

The  results  in  Verhagen's  study  were  obtained  through  an  experimental  procedure  that  follows  a  pattern  of 
presenting  video,  asking  questions,  and  giving  feedback  and  remediation.  This  basic  pattern  has  been  used 
by  other  researchers  who  studied  interactive  video  in  instruction  experiments  such  as  Schaffer  and  Hannafin 
(1986),  Phillips,  Hannafin,  and  Tripp  (1988),  and  Cennamo,  Savenye,  and  Smith  (1991),  The  procedure 
can  be  considered  as  sufficiently  similar  to  real  patterns  of  instruction  to  make  it  possible,  in  principle,  to 
reach  conclusions  with  respect  to  segment  length,  based  on  the  results  of  the  experiments,  that  can  be 
generalised  to  rules  for  the  design  of  segments  for  instructional  interactive  video  programs.  An 
unsatisfactory  aspect  of  the  reported  results  is,  however,  that  they  were  obtained  in  a  memory  experiment. 
The  performance  required  did  not  serve  any  instructional  purpose.  The  subjects  were  just  expected  to  recall 
every  factual  statement  as  presented  by  the  program,  although  literal  answers  were  not  necessary  (answers  to 
questions  that  showed  understanding  of  the  information  that  was  presented  by  the  pertaining  videodisk 
fragment  were  rated  to  be  correct).  The  observed  segment  lengths  may,  therefore,  not  represent  the  kind  of 
segment  lengths  that  are  effective  in  instructional  applications.  Looking  at  other  experiments,  an  obvious 
difference  concerns  the  information  load  per  segment  as  expressed  in  the  amount  of  questions  that  are 
presented  about  the  content  of  each  segment.  Schaffer  and  Hannafin  (1986),  for  instance,  worked  with  a 
video  program  of  13^/3  minutes  that  was  subdivided  into  five  segments.  The  mean  segment  length  of  their 
program  is  2.75  minutes.  The  information  load  per  segment  was  substantially  less  than  in  the  dense-packed 
videodisk  program  on  cheese  making.  Learners  had  to  answer  only  three  questions  per  segment  instead  of 
the  22  questions  per  segment  in  the  case  of  segments  of  3  minutes  in  Verhagen's  experiment. 

Schaffer  and  Hannafin  carried  out  their  experiment  with  ninth-through-twelfth  graders.  In  the  experiment, 
the  amount  and  type  of  interactivity  was  varied  in  four  conditions.  The  least  interactive  condition  was 
similar  to  the  Linear  Condition  of  Verhagen's  experiment:  just  presenting  the  video  program  without 
interruption.  In  the  most  interactive  condition,  next  to  the  five  video  segments,  questions,  feedback  and 
video  branching  were  components  of  the  interactive  structure.  Schaffer  and  Hannafin  used  a  measure  that 
they  called  "Acquisition  Rate".  Acquisition  Rate  was  derived  based  upon  the  ratio  of  recall  and  instructional 
time.  Schaffer  and  Hannafin  found  that:  "The  fully  interactive  version  yielded  the  greatest  recall  but  took 
longer  to  complete  than  any  of  the  other  presentations.  Time  was  shortest,  and  the  resulting  rate  of  learning 
was  greatest,  for  the  simple  linear  video  presentation"  (Schaffer  and  Hannafin,  1986,  p.  89).  Schaffer  and 
Hannafin  concluded  that  the  efficiency  of  interactive  video  for  teaching  factual  content  may  be  questionable. 
This  result  seems  to  show  similarities  with  Verhagen's  results,  specifically  with  the  relative  success  of  his 
Linear  Condition. 

Maccoby  r.i  "  Sheffield  (1961),  and  Margolius  and  Sheffield  (1961)  used  a  film  of  18  minutes  to  carry  out  a 
series  of  studies  about  segment  length  and  the  learning  of  a  mechanical-assembly  task.  They  introduced  the 
idea  of  "demonstration-assimilation  span"  (DA-span)  which  defines  a  kind  of  stopping  point:  "which  comes 
at  a  point  when  the  learner  has  assimilated  about  as  much  demonstration  material  as  he  can  hold  in  his  mind 
well  enough  to  translate  into  overt  practice  with  little  or  no  error"  (Maccoby  &  Sheffield.  196 1 ,  p.  79). 
Maccoby  and  Sheffield  and  also  Margolius  and  Sheffield  (1961)  used  fixed  segment  lengths.  A  DA-span 
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segment  was  defined  as  a  segment  which  could  be  practised  with  a  mean  group-performance  score  of  about 
75  per  cent  correctly  responding  immediately  after  viewing.  The  length  of  the  segments  differed 
considerably,  as  the  DA-span  was  determined  by  how  much  the  student  could  assimilate  in  one  stretch.  One 
of  the  four  sections  of  the  18-minute  film  had  a  running  time  of  six  minutes,  whereas  another  section,  a 
more  difficult  one,  ran  only  2.5  minutes,  while  both  were  considered  to  have  a  similar  DA-span  for  the 
given  target  group. 

The  interpretation  of  Anderson  and  Faust  (1975,  p.  215)  of  the  work  by  Sheffield  and  his  associates  was 
that  the  experiments  demonstrated  "that  short  presentations  followed  by  short  periods  of  practice  are  more 
effective  than  longer  presentations  followed  by  longer  periods  of  practice".  They  concluded  that:  "in 
summary,  experimental  evidence  seems  to  indicate  that  students  learn  more  when  required  to  make  active 
responses  immediately  after  small  segments  of  a  lesson"  (Anderson  &  Faust,  1975,  p.  216). 
The  tasks  in  these  experiments  were  very  different  from  just  learning  factual  information.  Students 
practised  the  assembly  task  with  concrete  parts  in  their  hands  and  cues  from  parts  already  assembled  or  still 
on  the  table.  The  order  of  magnitude  of  the  used  segment  lengths  for  the  assembly  task  is,  nevertheless, 
rather  similar  to  the  segment  lengths  from  the  studies  by  Verhagen  and  by  Schaffer  and  Hannafin. 
Although  the  experimental  procedures  showed  similarities,  differences  in  the  tasks,  content,  and  target 
groups  make  it  impossible  to  support  the  validity  of  Verhagen's  first  guideline. 

This  conclusion  is  reinforced  by  a  study  by  Weiss,  Maccoby,  and  Sheffield  (1961)  in  which  very  different 
segment  lengths  were  in  use.  Weiss,  Maccoby,  and  Sheffield  applied  the  same  research  methodology  that 
was  used  for  the  assembly  task  to  serial  learning  of  a  geometric-construction  task.  This  time  a  film  of  4.5 
minutes  was  used  that  demonstrated  the  nine-step  construction  of  a  five-sided  equilateral  polygon,  using 
pencil  and  paper  and  a  straight-edge  and  compass.  The  film  consisted  of  six  DA-span  segments.  The 
longest  DA-span  segment  in  this  film  was  75  seconds  in  length,  the  shortest  segment  was  12  seconds. 
They  concluded  that  "the  DA-span  unit,  as  the  practice  unit,  uniformly  gives  superior  performance  during 
practice  but  is  not  guaranteed  to  connect  together  the  different  parts  of  the  total  task.  A  compromise  which 
gives  at  least  some  practice  at  larger  units  of  the  task  may  be  desirable  even  if  it  entails  more  errors  in 
performance  at  the  out-set  of  practice."  (Weiss,  Maccoby,  &  Sheffield,  1961,  p.  76) 
In  the  geometry  task,  all  subsequent  steps  had  to  be  recalled.  This  requirement  resembles  more  the  subject 
of  the  other  studies  mentioned.  However,  the  used  segment  length  were  very  different.  Not  much  advice 
can  thus  be  derived  from  the  different  experiments  about  what  segment  length  should  be  preferred  for 
presenting  factual  information,  possibly  with  the  exception  that  segments  that  are  too  short  may  hinder  the 
integration  of  information  into  one  organised  whole  (Weiss,  Maccoby,  &  Sheffield,  1961).  It  is,  in  other 
words,  not  clear  whether  for  factual  information  some  frequently  alternating  presentation-and-practice  format 
should  be  developed  (which  complies  with  the  conclusion  of  Anderson  &  Faust,  1975)  or  whether  Webb's 
notion  that  lengthy  films  can  teach  well  (Webb,  1975),  and  may  yield  a  relative  efficient  learning  situation 
(Schaffer  and  Hannafin,  1986),  should  guide  designers. 


Research  questions 

The  research  that  is  reported  here,  was  designed  to  gather  further  insight  into  that  matter  and  was  specifically 
designed  to  find  out  whether  the  conclusions  of  Verhagen's  study  should  be  altered  if  data  collection  is 
carried  out  in  an  instruction  experiment  instead  of  in  a  memory  experiment.  The  second  guideline  is  also 
involved.  Verhagen's  contention  that  the  success  of  the  longer  segments  may  be  attributed  to  larger 
amounts  of  invested  mental  effort  (AIME),  was  reached  through  secondary  analysis  of  his  data.  In  his 
experiments,  no  measurement  of  AIME  had  been  done.  In  the  experiment  that  is  reported  here,  the  study  of 
the  effect  of  AIME  is  included. 

To  be  able  to  observe  differences  with  the  earlier  study,  the  same  videodisk  about  cheese  making  is  used. 
The  research  is  undertaken  to  attempt  to  answer  the  next  three  research  questions: 

Question  1: 

Given  an  experimental  videodisk  containing  25?.  information  elements,  does  self-chosen  segment  length 
differ  if  recall  of  factual  information  is  not  required  as  an  exercise  in  memory  performance  with  regard  to  the 
content  of  individual  information  elements  but  as  evidence  of  reaching  explicitly  stated  instructional 
objectives  concerning  the  program  as  a  whole? 


Question  2: 

What  differences  can  be  observed  between  self -chosen  and  program-controlled  segment  length  regarding  the 
investment  of  mental  effort  of  subjects? 
Question  3: 

How  do  the  differences  of  Question  2  relate  to  performance  on  post-tests  and  retention  tests? 


Instructional  format 

The  instructional  format  that  had  to  be  developed  for  the  experiments  was  not  very  different  from  the 
procedure  of  the  earlier  experiment.  The  basic  pattern  of  watching  video  segments,  answering  embedded 
questions  and  getting  feedback  on  the  answers  is  an  appropriate  strategy  to  get  active  and  learning  learners, 
as  is  demonstrated  by  research  of,  for  instance,  Schaffer  and  Hannafin  (1986),  Phillips,  Hannafin,  and  Tripp 
(1988),  and  Cennamo,  Savenye  and  Smith  (1991).  The  difference  is  that  the  content  of  the  videodisk  is 
analysed  to  serve  a  realistic  instructional  purpose  for  which  clear  instructional  objectives  are  formulated. 
The  orientation  of  the  learners  takes  place  on  the  basis  of  the  objectives  and  the  number  of  questions  have 
been  adjusted  to  the  to-be-learned  factual  knowledge  according  to  the  objectives.  The  total  ni  mber  of 
embedded  questions  in  the  measuring  section  of  the  new  version  of  the  program  is  97,  which  is 
substantially  less  than  the  213  questions  in  the  measuring  section  of  the  memory  experiment. 

The  orientation  of  learners  by  the  use  of  objectives  is  a  recommended  pre-instructional  strategy  before 
presenting  audio-visual  material,  as  are  pre-tests  and  overviews  (Morrison,  1979).  The  instructional  format 
has  been  shaped  accordingly.  A  list  of  objectives  was  made  available  to  the  learners  which  they  were 
required  to  study  before  working  with  the  interactive  video  program.  A  pre-test  is  introduced  in  the 
experimental  procedure.  The  control  of  the  program  is  organised  by  a  menu  structure  that  provides  an 
overview  of  treated  subjects  (Figure  lb).  This  approach  is  taken  despite  results  by  Phillips,  Hannafin,  and 
Tripp  (1988)  who  show  that  cueing  by  orienting  learners  with  introductory  statements  before  working 
through  video  segments  is  not  a  very  successful  way.  In  their  experiment  orienting  activities  of  this  kind 
accounted  for  less  than  3%  of  the  variance  of  post-test  scores,  while  their  findings  also  suggested  that 
cueing  by  practice  is  an  appropriate  strategy.  The  use  of  embedded  questions  may  thereby  serve  as  a  cueing 
strategy.  In  the  experiment  this  kind  of  cueing  is  provided  also.  The  orienting  activities  are  not  excluded, 
however,  because  other  research  does  support  Morrison's  recommendations.  Tovar  and  Coldevin  (1992),  for 
instance,  found  that  the  provision  of  orienting  activities  in  the  form  of  a  list  of  criteria  for  carrying  out  a 
certain  procedure,  a  slide  presentation  of  all  the  tools  used  in  this  procedure,  and  a  flow  chart  showing  the 
steps  of  the  procedure,  significantly  facilitated  the  recall  of  factual  information  from  an  interactive  videodisk 
lesson. 


The  experimental  program 

The  videodisk  program  that  was  used  for  the  study  was  the  same  program  that  was  used  in  Verhagen's 
experiments.  The  252  information  elements  of  this  program  were  divided  into  two  sections.  The  first  33 
information  elements  form  an  introduction  to  familiarise  the  subjects  with  the  experimental  procedure,  the 
remaining  219  information  elements  form  the  main  program  which  was  used  for  data  collection.  In  time, 
the  introduction  has  a  length  of  4.5  minutes  and  the  main  program  has  a  duration  of  31.5  minutes.  The 
controlling  computer  program  ensures  that  the  video  program  can  only  be  started  or  stopped  between  two 
information  elements.  This  causes  video  segments  of  the  program  always  to  consist  of  whole  numbers  of 
information  elements.  This  mechanism  was  applied  in  Verhagen's  experiments  (Verhagen,  1992a,  1992b) 
and  is  maintained  in  the  present  study.  In  the  experimental  conditions  that  were  developed  for  the  present 
study,  however,  the  last  three  information  elements  of  the  videodisk  program  are  not  used.  They  provide  a 
not-content-related  ending  of  the  linear  program  that  has  no  function  in  the  menu-driven  organisation  of  the 
new  conditions  that  are  described  below. 

In  the  main  condition  for  the  present  experiment,  the  measuring  section  was  structured  according  to  the 
following  menu  structure:  At  the  top  level  of  the  program,  the  Main  Menu,  the  subject  could  choose 
between  the  presentation  of  video  sequences  or  the  presentation  of  questions  related  to  the  diffe"  :nt  sequences 
(the  Main  Menu  is  presented  in  Figure  1  a). 
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In  the  Video-Sequence  Menu,  presented  in  Figure  lb,  the  subject  could  choose  one  cf  seven  sequences. 
Once  chosen,  the  video  started  playing.  The  subject  watched  the  sequence  until  he/she  decided  -  for  whatever 
reason  -  that  it  was  time  to  command  stopping  by  pushing  a  mouse  button.  The  program  consequently 
stopped  at  the  end  of  the  information  element  in  which  the  stopping  was  requested.  Then  a  menu  was 
presented  (Figure  Id)  in  which  the  subject  could  choose  to  go  on  with  the  presentation,  to  answer  questions 
or  to  return  to  the  Video-Sequence  Menu  (Figure  lb).  If  the  subject  chose  to  answer  questions  he/she  was 
presented  a  menu  (Figure  le)  in  which  the  subject  could  choose  to  answer  questions  about  the  segment 
he/she  just  watched,  to  answer  questions  about  the  entire  sequence  or  to  return  to  the  lasi  menu  (Figure  Id). 
At  the  end  of  a  sequence  the  subject  was  presented  a  menu  (Figure  1  f)  in  which  the  subject  could  choose  to 
repeat  the  sequence,  to  go  to  the  next  sequence,  to  answer  questions  or  to  return  to  the  Video-Sequence 
Menu.  If  the  option  to  answer  questions  was  chosen,  a  menu  was  presented  (Figure  lg)  in  which  the 
subject  had  options  to  answer  questions  about  the  last  segment  he/she  watched,  answer  questions  about  the 
whole  sequence,  go  to  the  Questions  Menu  (Figure  lc),  or  to  return  to  the  last  menu  (Figure  If)- 

If  the  subject  chose  to  answer  questions  about  a  segment  or  a  sequence,  all  embedded  questions  of  that 
segment  or  sequence  where  presented  successively.  All  questions  were  either  multiple  choice  questions  or 
short  answer  questions.  Next  feedback  was  given.  In  case  of  a  false  answer  the  subject  had  a  second  chance 
to  answer  the  question.  This  option  was  built  in  for  the  fact  that  subjects  tended  to  make  a  lot  of  typing 
errors  which  they  could  correct  at  this  point.  If  the  answer  was  still  not  correct  the  related  video  fragment 
was  repeated  followed  by  a  third  opportunity  to  answer  the  question.  As  soon  as  feedback  was  completed, 
the  subject  was  presented  the  next  menu.  After  a  stop  within  a  sequence  the  subject  could  choose  to  go  to 
the  beginning  of  the  next  segment  (which  starts  with  the  information  element  that  follows  the  one  in  which 
he/she  stopped)  or  to  return  to  the  Video-Sequence  Menu.  After  a  stop  at  the  end  of  a  sequence,  the  subject 
could  choose  to  go  to  the  beginning  of  the  next  sequence  or  to  return  to  the  Video-Sequence  Menu. 
If  the  subject  had  completed  the  viewing  of  a  whole  video  sequence,  the  related  menu  item  of  the  Video- 
Sequence  Menu  was  checked.  The  same  counted  for  the  questions.  If  all  embedded  questions  of  a  sequence 
were  answered  it  was  checked  in  the  Questions  Menu.  The  subject  had  to  answer  all  embedded  questions 
before  he/she  was  allowed  to  leave  the  program. 


Main  Men-  ;•! 

%  I    Presentalion  of  video  segues  ts  

|  Uuesi.jns  Menu  ~ 

i 
i 


|  Stop 


_  Make  your  choice 


Figure  1a.  Main  Menu, 


Video-Sequence  Menu 

1            Questions  Menu 

.5 

^Composition  of  the  milk 

Composition  of  the  milk 

3 

□  Preparation  of  the  cheesemakinrj  process           i }. 

i 

0  Preparation  of  the  cheesemaking  process 

| 

^Preparation  of  coagulation 

5 

□  Preparation  of  coagulation 

% 

□  The  coagulation  process 

1  K 

□  The  coagulation  process 

□  Operations  on  the  coaqulated  milk                    |  % 

3 

□  Operations  on  the  coagulated  milk 

□  Filling  the  cheese  moulds                            ;  1 

□  Filling  the  cheese  moulds 

□  Follow-up  treatments 

8 

□  Follow-up  treatments 

1 

J 

 ^  .  .  . 

Back  to  the  Main  Menu                                ;  5 

Back  to  the  Main  Men  u 

*J               Make  your  choice  $ 

%               Make  your  choice 

4 

Figure  1b.  Video-Sequence  Menu,* 


Figure  1c.  Questions  Menu,* 
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Figure  1d.  Menu  after  a  segment  not  being 
the  last  segment  of  a  sequence.. 


Figure  1e.  Questions  after  a  segment  not 
being  the  last  segment  of  a  sequence.. 
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Figure  If.  Menu  at  the  end  of  a 
sequence. 


Figure  1g.  Questions  after  the  last 
segment  of  a  sequence. 


'  The  menu  in  this  figure  is  presented  in  the  situation  that  the  first  and  third  sequence  have  been 

watched  entirely . 
'*  The  questions  of  the  first  sequence  have  been  answered. 


Figure  1:  Menu  structure  of  the  experimental  program  (main  condition) 
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The  distribution  of  information  elements,  objectives  and  questions  over  the  different  menu  items  is  listed  in 
table  1. 


Table  1 .  Distribution  of  information  elements,  objectives  and  questions. 


Number  of 

Sequences 

information 

objectives 

embedded 

elements 

questions 

Composition  of  the  milk 

25 

3 

12 

Preparation  of  the  cheese  making  process 

27 

8 

9 

Preparation  of  coagulation 

34 

7 

12 

The  coagulation  process 

25 

6 

20 

Operations  on  the  coagulated  milk 

35 

15 

17 

Filling  the  cheese  moulds 

29 

4 

8 

Follow-up  treatments 

41 

7 

19 

Total 

216 

50 

97 

The  data  presented  in  Table  1  only  account  for  the  main  program  (Information  Elements  34  to  249),  which 
was  the  measuring  section  of  the  program. 

The  33  information  elements  of  the  introduction  -  with  an  overview  of  the  cheese  making  process  - 
counted  25  embedded  question.  In  order  to  practice  the  menu  structure,  the  introduction  was  divided  into 
two  sequences. 


Subjects 

The  target  group  of  Verhagen's  experiments  consisted  of  university  freshmen.  This  target  group  is  atypical 
with  respect  to  the  educational  objectives  of  the  current  experiment.  Cheese  making  is  mainly  taught  on 
different  levels  of  agricultural  vocational  education.  The  target  group  of  the  current  experiments  is  therefore 
selected  on  the  middle  level  of  Dutch  agricultural  vocational  education.  Two  schools  participated  to  yield 
sufficient  subjects.  In  both  schools,  first-year  students  were  involved  in  one  of  two  programs:  a  three-year 
program  that  is  more  practice  oriented  (35  subjects)  and  a  four-year  program  that  aims  at  a  higher  level  of 
theoretical  background  than  the  three-year  program  does  (38  subjects).  Analysis  of  the  data  revealed  that 
with  respect  to  the  current  experiment  both  groups  showed  no  significant  differences  (Breman,  1995,  in 
Dutch).  In  this  contribution,  the  research  results  will  therefore  be  described  on  the  basis  of  the  pooled  data 
of  all  participants  in  a  certain  condition. 

Experimental  conditions 

EXPLORE: 

In  th;s  condition  the  subjects  were  familiarised  with  the  instructional  objectives  and  then  offered  the 
mc/.u-driven  environment  as  described  before,  to  explore  the  videodisk  by  choosing  video  sequences 
from  the  Video-Sequence  Menu  (Figure  lb)  or  by  entering  the  knowledge  domain  by  choosing  a  group 
of  questions  to  answer  from  the  Questions  Menu  (Figure  lc).  The  subjects  worked  through  the 
program,  dividing  each  video  sequence  if  so  desired  into  a  series  of  longer  or  smaller  segments 
according  to  the  procedure  that  was  explained  in  the  description  of  the  program.  The  subject  could 
decide  to  answer,  after  each  segment,  the  segment-related  questions  (and  then  get  feedback  and 
remediation)  or  could  choose  to  do  something  else  (such  as  interrupting  working  through  a  sequence 
and  return  to  a  menu  to  make  a  new  selection).  In  this  way,  a  situation  of  complete  learner  control 
existed,  with  the  already  mentioned  exception  that  all  questions  had  to  be  answered  before  leaving  the 
program. 

GUIDE: 

In  this  condition  the  subjects  were  familiarised  with  the  instructional  objectives  as  in  the  Explore 
Condition  and  then  were  offered  the  different  sequences  of  the  program  in  a  linear  order.  The  presented 
sequences  could  not  be  interrupted.  The  Guide  Condition  was  a  fixed-length  condition  in  which 
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presentation  of  videodisk  segments,  answering  questions,  and  remediation  in  case  of  incorrect  answers 
were  incorporated  in  a  tutorial  format  under  program  control. 

LINEAR: 

This  condition  resembles  Verhagen's  Linear  Condition  in  that  the  subjects  were  forced  to  watch  the 
whole  program  without  stopping.  In  this  case,  however,  the  subjects  were  informed  about  the 
instructional  objectives  beforehand  just  as  in  the  Explore  and  Guide  conditions.  Unlike  the  other 
conditions,  there  were  no  embedded  question  incorporated  in  this  condition. 

VARIABLE: 

Given  the  different  target  population  compared  to  university  freshmen,  the  main  condition  of  the  earlier 
experiment  by  Verhagen  ( 1992a,  1992b)  was  repeated  as  a  fourth  condition.  In  this  condition,  the 
subject  could  operate  the  program  in  a  way  similar  to  the  procedure  of  the  Explore  Condition. 
However,  no  menu  structure  was  used  ..nd  no  sequences  were  predetermined.  Instead  of  the  97 
objective-oriented  questions,  the  213  questions  of  Verhagen's  experiment  were  used.  The  questions  in 
Verhagen's  main  program  (which  was  the  measuring  section)  are  about  the  presented  factual 
information  that  is  contained  in  the  information  elements.  In  this  condition,  the  video  program  was 
presented  in  a  linear  way.  The  subjects  could  interrupt  the  program  by  clicking  a  mouse  button.  The 
subject  then  answered  all  questions  about  the  segment  just  viewed.  The  questions  were  all  open 
questions  that  required  short  sentences,  single  words,  or  numbers  to  be  typed  in  as  answer.  Next 
feedback  was  given.  Subjects  had  no  chance  to  correct  false  answers  before  remediation.  In  case  of  a 
false  answer  the  related  video  fragment  was  repeated  followed  by  a  second  chance  to  answer  the  question 
and  feedback  on  that  answer.  After  all  questions  of  the  segment  were  reviewed,  the  subject  was  allowed 
to  continue  the  program  beginning  with  the  information  element  that  followed  the  last  element  of  the 
completed  segment.  The  subject  had  control  over  the  length  of  each  video  segment,  but  no  control 
over  the  sequence  of  the  video  program. 


Research  method 

The  experimental  method  is  a  pre-test/post-test/retention-test  design. 

The  data  were  collected  during  the  September-October  period  of  1994.  Five  computer-videodisk-player 
combinations  were  available  (Olivetti  M240  and  M290  MS-DOS-computers  with  MCS  Video-overlay 
boards  and  Philips  videodisk  players  of  the  400  series).  The  experimental  sessions  thus  had  a  maximum  of 
five  subjects  per  session.  For  each  subject,  written  instructions  about  working  with  the  program  were 
available  for  reference,  together  with  a  pen  and  four  sheets  of  paper  for  making  notes,  and  a  study  guide  that 
contained  the  instructional  objectives  as  the  main  components  for  the  preparation  of  the  subjects  to  work 
with  the  program.  The  experiment  was  carried  out  according  to  the  following  procedure: 
The  experiment  was  introduced  by  a  brief  oral  presentation. 
Next,  a  paper-and-pencil  pre-test  of  25  multiple-choice  questions  was  administered. 
This  was  followed  by  an  introduction  of  the  subject  of  the  experiment  by  presenting  a  video 
segment  of  4.5  minutes  that  gives  an  overview  of  the  cheese  making  process.  This  introductory 
sequence  is  composed  of  the  first  33  information  elements  of  the  videodisk  program. 
In  the  learner-control  conditions  the  experimental  procedure  was  then  practised  by  working  through 
the  introductory  sequence  a  second  time  in  the  way  that  was  required  for  the  measuring  section  (the 
219  information  elements  of  the  main  program).  In  the  other  conditions  this  step  was  skipped. 
In  the  next  step,  the  subjects  were  invited  to  study  the  instructional  objectives  that  were  presented 
in  a  study  quide. 

A  questionnaire  was  administered  with  respect  to  the  expectancies  related  to  workin,"  with  the 
experimental  program.  The  subjects  answered  three  five-point-scale  questions  to  get  an  indication 
of  their  attitude  towards  the  program  (I  expect  to  like/dislike  working  with  the  program),  the 
Perceived  Demand  Characteristics  (PDC)  of  the  program  (What  I  have  to  learn  seems  difficult/easy 
to  me),  and  the  Perceived  Self  Efficacy  (PSE)  with  respect  to  the  task  (To  be  able  to  complete  the 
post-test  satisfactory,  1  will  have  to  invest  much/little  effort  during  working  with  the  program). 
After  all  these  preparatory  activities,  the  experimental  task  was  carried  out.  The  researcher 
remained  available  for  answering  questions  and  solving  mainly  minor  technical  problems  that  did 
occasionally  occur. 


611 

620 


After  completion  of  the  task,  the  subjects  were  allowed  to  go  one  time  more  through  their  notes 
and  the  list  of  instructional  objectives. 

The  next  step  was  administering  a  questionnaire  of  seven  or  eight  five-point-scale  questions 
(dependent  on  the  experimental  condition).  The  first  three  questions  were  similar  to  the  questions 
of  the  first  questionnaire.  The  question  "I  liked/disliked  working  with  the  program"  was  used  to 
find  out  whether  the  initial  attitude  did  change.  The  question  "What  I  had  to  learn  was 
difficult/easy"  was  again  intended  to  get  an  indication  of  the  PDC  of  the  program.  The  question  "I 
had  to  invest  much/little  effort  during  working  with  the  program"  was  used  to  get  an  indication  of 
the  Amount  of  Invested  Mental  Effort  (AIME)  to  complete  the  experimental  task.  The  other 
questions  mainly  concerned  the  audio-visual  presentation  of  the  program.  The  answers  revealed  no 
problems  with  the  quality  of  the  program  as  a  professional  audio-visual  product.  These  questions 
will  therefore  not  be  discussed  further.  More  information  can  be  found  in  Breman  (1995,  in 
Dutch).  One  question  asked  to  estimate  the  expected  success  in  making  the  post-test  (Now  that  I 
completed  the  task  with  the  program,  I  expect  that  1  will  perform  well/poorly  on  the  post-test). 
After  the  questionnaire,  a  paper-and-pencil  post-test  of  25  multiple-choice  questions  was 
administered. 

After  the  post-test,  a  third  and  last  questionnaire  was  administered,  this  time  consisting  of  four 
five-point-scale  questions.  One  question  concerned  the  test  performance  (Now  that  I  completed  the 
post-test,  I  think  that  I  did  well/poorly).  This  question  relates  to  the  question  about  the  expected 
success  in  making  the  post-test.  The  results  show  that  the  subjects  made  a  poor  estimate  of  their 
ability  to  perform  well  on  the  post-test,  while  their  judgement  of  their  performance  on  the  post- 
test  after  completing  it,  was  much  better.  For  the  pupose  of  the  present  study,  this  issue  needs  no 
further  attention.  The  other  questions  asked  for  preferences  with  respect  to  learning  from  books, 
video,  or  from  the  teacher.  The  results  of  these  questions  are,  again,  not  included  in  the  present 
contribution.  Detailed  results  are  available  from  the  authors. 
The  post-test  concluded  the  first  session. 

After  three  weeks  a  paper-and-pencil  retention  test  was  administered  that  again  had  25  multiple- 
choice  questions. 

Pre-test,  post-test  and  retention  test  we.-c  composed  from  a  pool  of  75  questions  that  consisted  of  25  groups 
of  three  questions.  The  three  questions  in  a  group  were  variants  of  one  and  the  same  question.  To  prevent' 
test  bias,  the  allocation  of  each  of  the  three  questions  in  a  group  to  either  the  pre-test,  the  post-test,  or  the 
retention  test  was  done  randomly  for  each  individual  subject.  Each  subject  thus  received  own  versions  of 
the  three  tests. 


Results 

Self-chosen  segment  length 

Table  2  and  Figure  2  show  the  differences  between  the  Variable  Condition  and  the  Explore  Condition  with 
respect  to  self-chosen  segment  length. 

The  mean  of  mean  self-chosen  segment  length  in  the  Explore  Condition  appears  to  be  more  than  twice  the 
length  of  the  mean  self-chosen  segment  length  in  the  Variable  Condition.  This  difference  is  significant 
(N=33,  F=23  68,  p<.0001,  tested  with  a  one-way  analysis  of  variance). 

A  possible  explanation  could  be  that  the  number  of  embedded  questions  was  very  different  in  these  two 
conditions.  To  be  able  to  reflect  on  that  difference,  segment  length  is  also  expressed  in  terms  of  the  number 
of  questions  that  the  subjects  had  to  answer  for  each  segment.  Mean  and  standard  deviation  (SD)  of  this 
measure  are  presented  in  the  two  most-right  columns  of  Table  2.  In  this  case,  no  significant  differences  are 
found  (N=33,  F=0.22,  p=.64,  tested  with  one-way  analysis  of  variance).  The  interpretation  of  this  fact  is, 
however,  difficult  because  of  the  unequal  distribution  of  embedded  questions  and  learning  objectives  over  the 
7  video  sequences  of  the  Explore  Condition  (Tabic  1). 
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Table  2:  Mean  of  mean  self-chosen  segment  length  by  condition. 

Mean  segment  length  expressed  in  terms  of 

Condition 

number  of  information  number  of  questions 
elements 

N    M            SD           M  SD 

Explore 
Variable 

20  21.55        7.75         9.50  3.37 
13  8.85         6.59         8.69  6.46 

Number  of  subjects 


8 


SI  VARIABLE 

■ 

1 
1 

1  ! 
1 
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Mean  self-chosen  segment  length 

Figure  2:  Distribution  of  self-chosen  segment  length 
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Test  performance  and  self-chosen  segment  length 

The  relationship  between  mean  self-chosen  segment  length  and  test  performance  is  analysed  by  correlating 
mean  self-chosen  segment  length  with  the  differential  scores  of  the  different  tests.  Table  3  shows  the 
correlations  of  the  Variable  Condition  as  well  as  of  the  Explore  Condition  with  the  differences  of  post-test 
and  pre-test  scores  and  of  retention  test  and  pre-test  scores.  For  this  analysis  the  non-parametric  correlation 
coefficient  'Kendall's  tau'  has  been  used.  Table  3  shows  that  there  is  no  reason  to  infer  that  mean  self- 
chosen  segment  length  affected  test  performance. 


Table  3:  Correlation  between  mean  self-chosen  segment  length  and  test  performance 


Post-test  minus 

Retention  test  minus 

pre-test  scores 

pre-test  scores 

N 

t  P 

t  P 

Condition: 

Explore 

Variable 

20 
13 

0.07  0.74 
-0.03  0.89 

-0.04  0.80 
-0.20  0.34 

Test  performance  and  condition 

Figure  3  shows  mean  test  scores  on  pre-test,  post-test  and  retention  test  for  each  of  the  experimental 
conditions.  Test  scores  are  absolute  scores.  The  maximum  test  score  was  25  in  all  cases. 
No  significant  differences  are  observed  with  respect  to  pre-test  scores  (N=73,  F=.22,  p=.88).  For  ihc  post- 
test  and  retention-test  scores,  the  data  of  Figure  3  are  not  suitable  to  analyse  differences  between  conditions. 
The  reason  is  that  pre-test  scores  varied  considerably,  ranging  from  3  to  16. 
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■  EXPLORE 

□  GUIDE 

■  LINEAR 

■  VARIABLE 

□  TOTAL 


i 

Pre-test 

Post-test 

Retention  test 

Condition 

N 

M 

SD 

M 

SD 

M 

SD 

EXPLORE 

20  , 

8.75 

3.14 

15.25 

4.04 

12.00 

3.10 

GUIDE 

18 

8.94 

2.48 

16.72 

4.03 

12.50 

2.81 

LINEAR 

18 

9.50 

3.03 

16.33 

2.00 

13.24 

2.02 

VARIABLE 

16 

8.94 

3.07 

15.15 

3.63 

12.46 

2.88 

TOTAL 

72 

9.03 

2.90 

15.90 

3.52 

12.53 

2.72 

Figure  3:  Mean  test  scores  per  condition 

The  assumption  that  the  subjects  had  no  pre-knowledge  about  cheese  making  appears  thus  not  to  be 
completely  true.  To  estimate  knowledge  gain,  differential  scores  are  used.  Figure  4  shows  the  mean  scores 
of  post-test  minus  pre-test  and  of  retention  test  minus  pre-test  for  each  of  the  conditions. ' 

■  EXPLORE 

□  GUIDE 
B  LINEAR 

■  VARIABLE 

□  TOTAL 

Post-test  -  Pre-test    Retention  test  -  Pre-test 
Figure  4:  Mean  differential  scores  per  condition 

Figure  4  shows  that  subjects  who  worked  in  the  Guide  Condition  gained  most,  on  average,  between  pre-test 
and  post-test.  Subjects  in  the  Linear  Condition  did  best  on  the  retention  test.  Two-way  analysis  of 
variance,  however,  revealed  no  significant  differences. 
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The  score  on  the  question  "I  had  to  invest  much/little  effort  during  working  with  the  program",  which  was 
part  of  the  questionnaire  that  was  administered  directly  after  the  completion  of  working  with  the 
experimental  program,  was  used  as  an  estimate  of  the  Amount  of  Invested  Mental  Effort  (AIME).  The 
score  on  the  question  "To  be  able  to  complete  the  post-test  satisfactory,  I  will  have  to  invest  much/little 
effort  during  working  with  the  program",  which  was  part  of  the  questionnaire  that  was  administered  before 
working  with  the  experimental  program,  was  used  as  an  estimate  of  Perceived  Self  Efficacy  (PSE).  The 
Perceived  Demand  Characteristics  (PDC)  of  the  program  were  rated  twice:  by  the  score  on  the  question 
"What  I  have  to  learn  seems  difficult/easy  to  me"  before  working  with  the  program  and  by  the  score  on  the 
question  "What  I  had  to  learn  was  difficult/easy"  after  working  with  the  program.  Table  4  shows  the 
correlations  (Pearson's  R)  between  AIME  and  the  other  measures.  The  significant  results  for  the 
relationship  between  AIME  and  PSE  suggest  that  the  subjects  were  honest  in  their  answers  and  indeed 
seriously  invested  effort  when  they  expected  that  the  difficulty  of  the  task  needed  it.  This  is  reinforced  by 
the  fact  that  more  significant  positive  correlations  emerged  between  AIME  and  PDC  after  working  with  the 
program  than  before  working  with  the  program.  The  experience  of  working  with  the  program  seems  to 
have  helped  the  learners  to  adjust  their  AIM*!  to  the  task. 


Table  4:  Correlations  of  AIME  with  PDC  and  PSE. 

AIME  versus: 

PDC  before 

PSE 

PDC  after 

R 

P 

R 

P 

R 

P 

ALL  SUBJECTS 

0.39 

<.001 

0.68 

<.001 

0.48 

<.001 

EXPLORE 

0.55 

0.01 

0.77 

<.001 

0.44 

0.05 

GUIDE 

0.18 

0.47 

0.61 

<0.01 

0.68 

<0.01 

LINEAR 

0.34 

0.16 

0.79 

<.001 

0.59 

0.01 

VARIABLE 

0.21 

0.45 

0.52 

0.04 

0.00 

1.00 

Table  5  shows  that  t!.e  subjects  adjusted  their  PDC  due  to  working  ■ 4n  the  program.  In  three  conditions,  a 
significant  positive  correlation  seems  to  imply  that  the  initial  PDC  was  in  most  cases  a  fair  estimate.  In 
the  Variable  Condition  this  was  clearly  not  the  case.  The  reason  could  be  that  practicing  the  experimental 
procedure  with  the  introduction  was  eventually  not  perceived  as  representative  for  the  task  in  the  measuring 
section.  The  data  showed  that  in  this  condition,  PDC  was  rated  to  be  slightly  more  difficult  before  working 
with  the  program  than  after  working  with  the  program. 


Table  5:  Correlations  of  PDC  before  working  with  the  program 
with  PDC  after  working  with  the  program. 


PDC  after  versus: 

PDC  before 

R  P 

ALL  SUBJECTS 

0.46  <.001 

EXPLORE 

0.73  <.001 

GUIDE 

0.56  0.02 

LINEAR 

0.46  0.06 

VARIABLE 

0.04  0.89 

The  effect  of  AIME  on  test  performance  was  analysed  using  a  Kruskal-Wallis  one-way  analysis  of  variance 
(Table  6).  No  significant  relationships  were  discovered.  In  the  Explore  Condition  a  light  tendency  towards 
some  influence  of  AIME  is  visible. 


Table  6:  Differences  in  performance  by  AIME  scores  for  the 

different  conditions.  

AIME  scores  versus: 

post-test  minus  retention  test  minus 


pre-test  scores 

pre-test  scores 

Chi2 

P 

Chi2 

P 

ALL  SUBJECTS 

3.13 

0.37 

1.37 

0.71 

EXPLORE 

5.35 

0.15 

5.90 

0.12 

GUIDE 

0.36 

0.95 

2.13 

0.55 

LINEAR 

2.53 

0.47 

3.13 

0.37 

VARIABLE 

0.36 

0.83 

1.05 

0.59 

The  difference  of  AIME-Scores  between  condition  showed  some  significance  (Table  7).  Analysis  of  the 
data,  using  the  Mann-Whitney  U-Wilcoxon  Rank  Sum  W  test  corrected  for  ties,  showed  that  for  working 
with  the  Explore  Condition  significant  less  mental  effort  was  invested  than  in  the  Linear  Condition  (P=.05) 
or  in  the  Variable  Condition  (P=.01).  Other  relationships  were  not  significant,  although  the  Guide 
Condition  shows  a  tendency  to  have  required  less  effort  than  the  Variable  Condition  (P=.09). 

Table  7:  Difference  in  AIME-Scores  between  conditions.  

Conditions  

Chi2  P 
AIME  S\07  0.05 


Discussion 

In  a  first  instance,  the  mean  of  mean  self-chosen  segment  length  in  the  Variable  Condition  (mean  of  mean 
length:  8.85)  seems  to  differ  from  the  results  from  Verhagen's  earlier  experiment.  However,  the  mean  of 
mean  segment  length  in  his  study  (1 2.70  Information  Elements)  was  in  itself  the  mean  of  the  results  from 
three  different  groups.  They  were  university  students  from  engineering  departments  (mean  of  mean  length: 
17.66  Information  Elements),  students  from  a  faculty  of  public  administration  (mean  of  mean  length:  14.07 
Information  Elements),  and  students  form  a  faculty  of  educational  science  and  technology  (mean  of  mean 
length:  7.42  Information  Elements).  The  results  of  the  present  study  compare  to  the  results  of  the  last 
group  and  can  thus  be  regarded  to  fall  within  the  normal  range  of  the  experimental  method  of  the  earlier 
study. 

The  difference  between  the  Variable  Condition  and  the  Explore  Condition  seems  therefore  to  be  attributable 
to  task  differences.  Although  the  subjects  in  the  Variable  Condition  were  oriented  on  the  experimental  task 
by  the  pre-test  and  the  list  of  learning  objectives  just  as  the  subjects  in  the  other  conditions  were,  it  seems 
likely  that  the  large  number  of  very  specific  questions  in  this  condition  did  shape  their  behaviour.  They 
adopted,  in  other  words,  a  strategy  for  working  through  the  program  that  compares  to  the  university 
students  in  the  earlier  experiment  by  Verhagen  (1992a).  The  result  that  AIME  and  PDC  showed  hardly  any 
or  no  relationship  in  the  Variable  Condition  (Table  4  and  Table  5)  may  be  interpreted  to  support  this  view. 
In  all  other  conditions,  the  task  requirements  were  consistent  with  the  orienting  activities.  In  these 
conditions,  embedded  questions  were  designed  to  relate  to  the  explicitly  stated  learning  objectives.  Working 
with  the  program  thus  helped  to  clarify  PDC.  The  own  set  of  questions  of  the  Variable  Condition  and  the 
relatively  rigid  experimental  procedure  in  this  condition  may  have  caused  interference  in  that  respect. 
The  answer  to  the  first  research  questions  seems  to  be  positive.  The  task  structure  of  the  Variable 
Condition,  which  was  in  essence  an  exercise  in  memory  performance,  lead  to  segment  lengths  that  were,  on 
average,  four-tenths  of  the  length  of  the  self-chosen  segments  in  the  Explore  Condition,  which  was  an 
instructional  environment  under  learner  control. 

No  evidence  was  found  that  test  performance  was  related  to  self-chosen  segment  length  (Table  3). 
Differences  in  test  performance  between  conditions  were  small.  Subjects  who  worked  in  the  Guide 
Condition  did  best  on  the  post-test.  Subjects  who  worked  in  the  Linear  Condition  did  best  on  the  retention 
test.  These  results  are  similar  to  the  results  of  Verhagen  (1992b).  Mean  length  of  the  fixed  segments  in 
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the  Guide  Condition  was  30.86  Information  Elements  (in  time:  about  4,5  minutes),  while  the  mean  of 
mean  self-chosen  segment  length  in  the  Explore  Condition  was  21.55  Information  Elements  (in  time:  about 
3  minutes).  Also  Verhagen  found  that  forced  watching  of  relative  long  segments  supports  learning. 
Whether  this  implies  more  invested  mental  effort,  however,  is  not  clear.  The  self-reported  AIME  by  the 
subjects  does  not  show  any  relationship  with  test  performance  (Table  6).  With  respect  to  the  Linear 
Condition:  In  Verhagen's  case  this  condition  also  outperformed  all  conditions  on  the  retention  test.  He 
suggested  that  the  fact  that  subjects  who  watch  the  linear  version  are  provided  with  an  overview  of  the 
whole  program,  may  support  later  recall.  The  data  from  Breman  (1995)  show  that  in  the  Linear  Condition 
the  subjects  completed  the  experimental  task  in  about  half  of  the  time  that  was  needed  for  the  Explore 
Condition  or  the  Guide  Condition.  The  Variable  Condition  took  even  more  time.  Breman  further  found 
that  after  working  with  the  program,  the  subjects  in  the  Linear  Condition  liked  the  program  better  than  the 
subjects  in  any  of  the  other  conditions.  These  facts,  combined  with  the  positive  results  of  the  Linear 
Condition  regarding  retention-test  performance,  suggest  that  when  video  is  considered  for  instruction  with 
the  acquisition  of  factual  knowledge  as  its  main  goal,  linear  video  should  be  preferred  instead  of  interactive 
video.  This  conclusion  is  further  supported  by  the  conclusion  of  Schaffer  and  Hannafin  (1986)  who  found 
that  linear  video  had  the  highest  Acquisition  Rate. 

The  second  research  questions  asks  what  differences  can  be  observed  between  self-chosen  and  program- 
controlled  segment  length  regarding  the  investment  of  mental  effort  of  subjects.  The  answer  is  found  in 
Table  7.  According  to  the  self-reports  by  the  subjects,  for  working  with  the  Explore  Condition  significant 
less  mental  effort  was  invested  than  in  the  Linear  Condition  or  in  the  Variable  Condition.  The  Guide 
Condition  shows  a  tendency  to  have  required  less  effort  than  the  Variable  Condition.  The  Explore 
Condition  and  the  Guide  Condition  are  new  conditions  that  have  been  developed  to  provide  a  well-designed 
learner-controlled  or  program  controlled  system,  respectively.  Both  conditions  appear  to  offer  the  most 
relaxed  learning  situation  and  --  if  comfort  may  be  taken  as  a  criterion  -  the  best  usability.  This  usabilty 
aspect  does  not  support  the  preference  for  linear  video  that  was  concluded  before. 

That  the  Guide  Condition  requires  more  AIME  than  the  Explore  Condition  is  probably  due  to  the  fact  that 
the  mean  length  of  the  fixed  segments  in  the  Guide  Condition  was  about  1,5  times  the  mean  of  mean  self- 
chosen  segment  length  in  the  Explore  Condition.  It  would  be  interesting  to  know  how  data  would  change 
if  the  mean  length  of  the  fixed  segments  in  the  Guide  Condition  would  be  adjusted  to  be  similar  to  the 
mean  length  of  the  Explore  Condition.  Would  AIME  be  reduced?  Would  then  the  better  performance  on 
the  post-test  also  be  reduced?  A  clear  recommendation  which  of  these  conditions  should  be  preferred  is,  in 
any  case,  not  possible  on  the  basis  of  the  present  data. 

The  third  research  question  concerns  the  relationship  between  AIME  and  test  performance.  No  such 
relationship  could  be  demonstrated  (Table  6). 


Conclusion 

Different  approaches  for  segmenting  video  and  organising  interaction  appear  to  yield  different  results  with 
respect  to  test  performance,  instruction  time,  AIME,  and  liking  of  a  learning  task  that  concerns  the 
acquisition  of  factual  knowledge.  Linear  video  shows  some  advantages  for  learning  of  factual  knowledge. 
A  clear  preference  for  linear  video  over  interactive  video  is,  however,  not  indicated  in  all  circumstances.  On 
the  basis  of  the  results  of  the  present  experiment,  the  amount  of  mental  effort  that  is  required  for  relatively 
long  linear  programs  seems  to  be  more  than  in  interactive  programs  in  which  the  video  is  segmented  in  a 
proper  way.  In  situations  where  next  to  learning  factual  knowledge  also  more  complex  learning  outcomes 
are  desired,  the  advantages  of  linear  video  may  remain  valid  for  the  factual  components  of  the  instruction.  A 
well-designed  interactive  video  application  may  offer  a  comfortable  work  environment  that  allows  for  a 
relaxed  use  of  mental  effort.  The  results  of  the  experiment  seem  to  suggest  that  video  segments  with  a 
length  of  about  3  minutes,  containing  enough  factual  information  for  answering  8  or  9  questions,  may 
appear  to  be  optimal  building  blocks.  Subsequent  experiments  should,  however,  reveal  to  what  extent  this 
suggestion  is  valid. 
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Introduction 


Enterprises  of  all  sorts  tend  to  develop  modes  of  operation  that,  over  time,  come  to  be  accepted  as 
the  "conventional  way."  Education  has  always  relied  heavily  on  one  format:  the  classroom  model. 
Distance  education—which,  in  its  early  years,  was  likely  seen  as  an  unconventional  way  to  facilitate 
learning— developed  its  own  conventional  model:  the  "correspondence  course."  And  despite  continuous 
progress  in  developing  alternative  means  of  communication,  these  conventional  formats  have  been  slow  to 
give  way  to  innovative  approaches.  Now,  in  the  face  of  what  Lewis  Perelman  (1992)  calls  "an  implacable 
technological  revolution,"  that  appears  to  be  changing.  What  this  revolution  entails  and  how  it  may  affect 
education,  and  distance  learning  in  particular,  are  critical  questions  for  educators  everywhere. 

This  paper  therefore  "revisits"  distance  learning  by  addressing  its  past  achievements,  its  present 
state,  and  its  future  in  the  face  of  the  rapidly  converging  computer  and  communications  technologies  and  the 
goals  and  potential  that  underlie  the  creation  of  the  proposed  National  Information  Infrastructure  (Nil).  The 
analysis  was  undertaken  recognizing  that  new  demands  are  being  placed  on  our  educational  institutions  to 
provide  a  highly  knowledgeable  and  skilled  workforce  for  the  Twenty-First  Century;  that  the  stakes  involved 
are  no  less  than  our  economic  competitiveness— which  is  now  so  knowledge  dependent;  and,  that 
consideration  of  the  present  and  future  of  distance  learning  requires  a  "paradigm  shift"  in  the  role  of  teachers 
and  educational  institutions  as  the  United  States  enters  into  a  global  economy. 

This  inquiry  therefore  seeks  to  create  a  contemporary  vision  of  distance  learning  based  on  1)  its 
history,  2)  the  recognition  of  the  importance  of  life-long  learning  and  the  development  of  new  strategies  for 
its  realization;  3)  the  potential  of  emerging  telecommunications  technologies;  4)  the  respective  roles  of 
private  industry  and  government  in  the  creation  of  the  National  Information  Infrastructure;  and  5)  the 
potential  contribution  of  The  High  Performance  Computing  and  Communications  (HPCC)  initiative.  We 
began  to  construct  this  vision  by  examining  efforts  to  define  distance  education  and  to  establish  its 
theoretical  base,  distance  education's  major  milestones,  technology's  expanding  role  and  potential,  and  new 
opportunities  that  derive  from  the  National  Information  Infrastructure  Initiative. 


Distance  Education  Versus  Distance  Learning 

A  definition  that  accurately  describes  an  activity's  nature  and  function  is  essential  forjudging  its 
history  and  creating  a  vision  of  its  future.  Schlosser  and  Anderson  (1994)  state  that  the  term  "distance 
education"  has  been  used  to  describe  a  great  variety  of  programs  employing  a  wide  variety  of  media.  And 
they  cite  several  scholars  who  have  expanded  on  distance  education's  key  element-bridging  the  physical 
separation  that  exists  between  teacher  and  student  via  some  medium:  Rudolf  Delling  emphasized  that 
distance  education  is  "a  planned  and  systematic  activity  [that  involves]  didactic  preparation  and  presentation 
of  teaching  materials  as  well  as  the  supervision  and  support  of  student  learning...;"  Greville  Rumble 
declared  that  distance  education  involves  a  "contract"  between  the  student  and  the  teacher,  or  teaching 
institution,  that  defines  respective  roles;  and  Desmond  Keegan  noted  that  despite  the  separation  of  teacher 
and  student,  the  influence  of  an  educational  institution,  the  provision  "f  two-way  communication  and  the 
possibility  of  occasional  meetings  all  distinguish  distance  education  •rom  private  study  .  Moreover,  in 
distance  education,  much  communication  occurs  noncontiguously. 

These  points  are  well  made.  But,  to  simplify,  here  we  use  the  term  "distance  education"  to  indicate 
the  organizational  apparatus  and  process  for  providing  educational  experiences  to  people  at  a  distance,  and 
the  term  "distance  learning"  to  refer  to  the  process  and  result  of  attending  to,  and  responding  to,  such 
experiences.  As  future  developments  in  both  communication  and  knowledge  access  technologies  provide 
learners  with  increased  capability  to  learn  independently,  we  expect  the  distinction  between  distance 
education  and  distance  learning  to  become  more  meaningful. 


Distance  Education  Milestones 

Distance  education's  history  is  more  than  a  catalog  of  pre  ram  growth  and  technology  evolution, 
although  that  is  part  of  it.  Schlosser  and  Anderson  (1994)  trace  distance  education's  roots  in  the  United 
States  to  1873  when  correspondence  study  "crossed  the  Atlantic"  from  Europe  and  The  Society  to  Encourage 
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Studies  at  Home  was  founded  in  Boston.  Other  programs  followed  as  correspondence  study  continued  to 
expand  into  the  twentieth  century. 

By  the  1920s,  electronic  technology  had  become  a  major  factor  in  distance  education  with  at  least 
176  radio  stations  being  built  at  educational  institutions.  Most,  however,  did  not  survive  the  decade 
(Schlosser  and  Anderson,  1994).  Some  did  remain,  however,  and  by  the  1950s,  a  second  broadcast  medium, 
television,  began  to  play  a  growing  role  in  distance  education.  Western  Reserve  University  and  New  York 
University,  with  its  well-known  Sunrise  Semester  courses  on  CBS.  were  early  users  of  the  medium.  Later, 
as  satellite  television  became  cost  effective,  television's  role  expanded.  The  first  state  educational  system, 
Learn/Alaska,  began  operations  in  1980. 

Schlosser  and  Anderson  (1994)  mark  the  1962  decision  by  the  University  of  South  Africa  to 
become  a  distance  education  institution  as  the  beginning  of  a  "fundamental  change  in  the  way  distance 
education  was  practiced  in  much  of  the  world."  The  founding  of  the  Open  University  of  the  United 
Kingdom  in  1971  was  another  landmark.  As  a  degree-granting  institution  dedicated  to  distance  teaching,  it 
pvtended  the  university  opportunity  to  many  and  brought  added  prestige  to  distance  education  as  an 
enterprise.  The  Open  University  offered  degree  programs  in  many  fields  and  made  wide  use  of 
communication  media.  Similar  institutions  soon  followed  in  Canada,  Japan  and  West  Germany,  although 
not  all  embraced  the  Open  University's  liberal  enrollment  policy. 

A  good  measure  of  distance  education's  recent  growth  is  the  number  and  variety  of  programs 
presently  available.  Peterson's  Guides  1993  publication  The  Electronic  University  is  one  excellent  source 
of  information  on  "telecourses"  and  degree  programs  offered  "electronically"  to  US  and  Canadian 
inhabitants.  It  describes  nearly  one  hundred  institutional,  network  or  consortia-based  programs.  Well 
known  consortia  currently  offering  distance  learning  programs  include  the  International  University 
Consortium  (IUC),  Mind  Extension  University  (ME/U),  the  National  Technological  University  (NTU),  and 
the  Electronic  University  Network  (EUN). 


Technology's  Growing  Influence  On  Education 

Technology,  as  it  advances,  is  playing  a  larger  role  in  both  school-based  and  distance  education.  In 
schools  we  have  seen  technology  use  evolve  from  "audiovisual  aids"  used  to  support  classroom  lectures  to 
various  forms  of  media-intensive  learning  centers  and,  more  recently,  to  modern  interactive  technology 
systems.  In  distance  education,  educational  radio— which  expanded 

greatly  after  1925  (Saettler,  1968)— was  followed  by  broadcast  television  and  teleconferencing  as  major 
delivery  innovations. 

An  interesting  study  by  Wilkinson  and  Sherman  (1991)  provides  a  useful  portrait  of  distance 
programs  then  active.  They  contacted  276  higher  education  distance  education  programs  and  received  142 
"usable"  survey  responses.  With  regard  to  the  primary  method  of  delivery  for  various  courses  across  all 
institutions,  video-based  technology  was  the  most  frequently  used  method  with  about  three-fourths  of  the 
programs  using  video  in  more  than  half  of  their  courses  (p.  56).  Print-based  instructional  delivery  was 
second  in  frequency  of  use  with  both  audio-  and  computer-based  technology  enjoying  lower  usage.  Print, 
however,  was  the  primary  method  of  student/faculty  interaction  in  over  75  per  cent  of  courses  offered. 
Clearly,  most  institutions  employ  a  variety  of  media  to  conduct  their  courses. 

The  marriage  of  computers  and  communication  media  has  led  to  new  forms  of  in-school  learning 
environments -usually  some  form  of  learning  station  or  desktop  application-that  both  supplement  and 
replace  standard  classroom  experiences.  In  distance  learning,  use  of  teleconferencing  tools  that  enable  two- 
way  communications  is  increasing.  And  networks  now  enable  educators  and  learners  to  communicate 
through  e-mail  and  bulletin  boards  and  to  collaborate  on  problem  solving  tasks. 

Thus,  as  the  power  and  versatility  of  computer-mediated  communication  is  amplified  by  new  tools 
with  enormous  capacities  to  store,  transmit  and  control  information,  technology  can  offer  educators  a  broad 
selection  of  interactive  learning  systems.  These  include  desktop  interactive  multimedia,  electronic 
classrooms  and  other  local-area  network  applications,  wide-area  network  and  teleconferencing  systems, 
virtual  reality  environments  and  "microworlds."  And  the  proposed  National  Information  Infrastructuic  (Nil) 
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and  "intelligent  assistants"  (Roesler  and  Hawkins,  1994)  are  reportedly  just  over  the  horizon.  So  while  the 
term  "information  highway"  may  be  a  poor  technical  metaphor,  there  is  an  analogy  related  to  its 
construction  with  which  educators,  and  the  "highway's"  builders,  can  relate.  The  interstate  system  was 
envisioned  in  the  1950s  as  a  new  kind  of  road  (a  "superhighway")  that  would  provide  greater  traffic  volume 
and  increased  speeds.  The  Nil,  when  completed,  will  expand  information  traffic  in  much  the  same  way. 
For  distance  educators,  and  the  computer,  telephone  and  cable  industries,  it  is  the  1950s  again;  a  time  to 
make  long-range  plans—in  terms  of  both  distance  and  time.  Indeed,  as  distance  learning  programs  are 
helping  to  recast  the  traditional  campus,  newer  technologies  are  promising  to  recast  distance  learning  itself. 
It  truly  may  be,  as  The  Electronic  University  proclaims,  "the  hottest  education  trend  of  the  90s." 


Distance  Learning  in  Transition 

Distance  education's  ongoing  efforts  to  employ  new  technologies  to  reach  learners,  to  incorporate 
new  teaching  methods  and  to  create  new  administrative  forms  has  placed  distance  learning  in  a  definite  state 
of  transition. 


Distance  Learning:  The  Current  Picture 

From  a  near-totai  reliance  on  print-based  courses,  technology  use  in  distance  learning  expanded  to 
include  broadcast  radio  and  television  and,  later,  audio,  video  and  computer  teleconferencing.  Now 
computer-based,  multimedia  teleconferencing  systems  are  the  emerging  trend  (Romiszowski,  1993).  Such 
sophisticated  new  technologies  are  presently  shifting  the  emphasis  "from  single-technology  delivery 
systems  to  integrated  approaches  seeking  to.. .combine  voice,  video,  and  data  technology,  often  in  tandem 
with  print"  (Willis,  1993,  p.  10). 

With  respect  to  methodology,  many  distance  educators  seem  to  describe  and  gauge  the  instructional 
models  they  employ  in  relation  to  conventional  classroom  practice.  Correspondence  study  suffers  in  this 
perspective;  it  offers  no  live  Communication.  Open  broadcast  radio  and  television  offer  the  lecture  aspect  of 
the  classroom,  but  not  the  interaction  opportunities.  The  early  teleconferencing  systems  offered  limited 
learner  participation,  mainly  through  telephone  communication.  The  new  computer-mediated 
telecommunications  systems  are  being  touted  highly  because  they  promise,  via  "real-time"  multimedia 
interactive  communication,  the  "virtual"  equivalent  of  the  regular  classroom  experience. 

It  would  be  a  mistake,  however,  to  limit  the  concept  and  practice  of  distance  learning  to  the  simple 
notion  of  delivering  conventional  classroom  instruction  at  a  distance.  This  would  "short  circuit"  the 
potential  of  telecommunications  technology  to  transform  education  as  we  know  it  into  a  process  for 
enabling  children  and  adults  of  all  ages  to  empower  themselves  and  to  become  viable  members  of  the 
emerging  global  society  and  economy.  Distance  learning  must  provide  opportunities  to  acquire  knowledge 
and  skills  both  during  one's  formal  education  years  and  on  a  "just  in  time"  basis  throughout  life  to  meet  the 
constantly  changing  requirements  for  adult  productivity  in  our  society. 

The  question  of  what  institutional  structures  are  needed  to  accommodate  distance  learning  activities 
has  become  critical.  Although  university  level  distance  education  has  grown  steadily  and  upgraded  its 
technology  component  throughout  the  twentieth  century,  from  an  institutional  perspective  it  remained  for  a 
long  time  what  Michael  Moore  (199-3)  calls  a  "first  generation"  model.  That  is,  distance  students  received, 
either  by  correspondence  or  by  telecommunications,  "classroom-like  teaching.. .provided  by  conventional 
teachers...."  The  1969  opening  of  Britain's  Open  University  as  an  institution  specifically  dedicated  to 
creating  and  delivering  distance  education  programs-and  of  similar  institutions  in  other  countries-was  a 
major  milestone  (Willi:   1993).  These  "second-generation  distance  education  institutions,"  Moore  (1993) 
observed,  "had  no  classrooms  [since]  learning  occurred  in  students'  own  environments."  Faculty  were 
chosen  specifically  for  distance  teaching  and  the  institutions  were  given  the  authority  to  grant  their  own 
degrees.  But  although  the  Open  University  model  breaks  the  tradition  that  requires  teacher  and  students  to 
meet  in  a  classroom  for  all  educational  transactions,  it  shares  with  conventional  schools  one  very  important 
feature:  courses  are  developed  and  taught  and  achievement  is  certified  by  the  faculty  associated  with  a 
particular  place.  Students  therefore  are  as  limited  in  their  selections  of  courses  and  teachers  as  they  are  at  a 
conventional  "brick-and-mortar"  campus. 
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Recent  technology  developments  promise  to  remove  such  limitations.  We  are,  Moore  (1993) 
suggests,  "rapidly  approaching  technical  readiness  for  the  Virtual  University,  the  third  generation  of  higher 
distance  education."  In  such  universities,  courses  developed  anywhere  are  made  available  to  students 
everywhere,  as  is  wide  access  to  libraries  and  other  resources  for  information  and  guidance.  Moore's 
prescience  was  demonstrated  in  late  1994  with  the  advent  of  the  Virtual  Online  University  (VOU).  It  offers, 
on  the  Internet,  a  virtual  education  environment  that  combines  the  academic  environment  of  a  physical 
world  university  plus  the  creative  dimension  of  a  Multi-User  Domain  or  MUD. 

Issues  in  Distance  Learning 

There  is,  however,  in  the  creation  of  a  viable  regional,  national  or  global  distance  learning 
organization,  more  to  be  decided  than  what  technology  will  be  employed  and  how  it  will  be  used.  As 
Rossman  (1993)  asserts:  "The  agenda  for  global  education  begins  with  questions  about  who  is  to  coordinate 
and  regulate  electronic  courses  offered  on  network  or  satellite;  who  is  to  set  standards,  especially  when 
nations  and  universities  disagree;...and  who  is  to  arbitrate  and  decide  on  such  matters  as  degrees  and 
exchange  of  course  credits"  (p.  13).  And  as  various  state  systems  and  institutions  expand  distance  learning 
offerings,  we  can  expect  numerous  policy  and  administrative  issues  to  come  to  the  fore. 

Thus  reaching  a  state  of  "technical  readiness"  for  what  Rossman  (1993)  calls  The  Emerging 
Worldwide  Electronic  University,  may  be  easier  than  reaching  an  equivalent  state  of  readiness  in  the 
curricular  and  administrative  areas.  Moore  (1993)  cites  the  major  administrative  problem:  "In  spite  of  a 
number  of  efforts  to  establish  certification  of  learning  by  non-traditional  methods,  there  is  still  no  agency  in 
North  America  that  has  the  power,  recognition,  or  authority  to  provide  a  certification  of  independent, 
individually  constructed  learning  programs  that  has  credibility  in  the  academic  and  business  environment 
comparable  to  the  degree  awarded  by  a  brick-and-mortar  university"  (p.  6).  And  the  problems  that  typically 
accompany  the  development  and  introduction  of  innovative  educational  programs  are  likely  to  be  amplified 
in  the  distance  learning  arena.  Jacobson  (1994)  recently  identified  several  important  issues  associated  with 
extending  the  reach  of  "virtual"  classrooms,  including:  1)  the  effect  of  distance  learning  on  faculty 
workload/compensation;  2)  faculty  involvement  in  course  design;  3)  faculty  job  security;  4)  certification  in- 
state and  across  states;  and  5)  concern  that  distance  learning  becomes  more  than  simply  transmitting  lecture., 
from  one  place  to  another. 

So  as  advancing  technology  affects  perceptions  of  what  instructional  outcomes  can  be 
accomplished  at  a  distance,  a  number  of  vital  questions  follow:  Do  current  definitions  of  and  theories  about 
distance  education  provide  an  adequate  basis  for  exploiting  the  potential  of  future  distributed  learning 
structures?  Or  do  they  matter  at  all?  Are  the  design  principles  of  conventional  education  adequate  for 
developing  distance  learning  programs?  Are  institutions  up  to  the  task?  What  are  the  respective  roles  of 
teachers  and  learners  in  both  short-term  and  life-long  distributed  learning  applications?  How  should  distance 
learning  designs  respond  to  current  trends  in  American  professional  and  industrial  settings  that  are 
transforming  work  and  models  of  learning? 

Perhaps  more  than  ever  before,  distance  learning  remains  an  enterprise  in  transition,  a  transition 
driver,  by  advances  in  technology,  recognition  of  the  need  to  improve  the  quality  of  education,  a  growing 
demand  on  the  part  of  various  student  populations  for  increased  learning  opportunities,  and  the  need  to  make 
maximum  use  of  available  resources.  With  so  many  forces  affecting  distance  learning's  future,  it  seems  fair 
to  say  that  we  know  better  where  the  technology  is  going  than  we  know  where  we  are  going  with  the 
technology.  Why?  Mainly  because  the  "enabling  technologies'"  required  to  create  a  national  or  global 
information  "infrastructure"  are  known,  their  past  development  has  been  carefully  noted,  and  their  projected 
development  in  the  years  immediately  ahead  is  fairly  predictable.  This  applies  to  computer  chips,  which 
provide  processing  power,  to  storage  devices,  which  promise  massive  increases  in  capacity,  to 
optoelectronics,  v/hich  is  increasing  communication  speed  ,  to  advanced  switching  devices  needed  to  blend 
voice,  video  and  data,  to  digital  compression  techniques,  and  to  software  technology,  which  is  moving 
steadily  toward  the  goal  of  providing  so-called  "intelligent"  agents  to  help  users  manage  the  information 
available  to  them. 

But  how  competition  in  the  commercial  world  of  communications  companies  will  play  out,  what 
role  the  government  will  assume,  what  initiatives  various  educational  institutions  will  launch,  and  what 
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learning  opportunities  students  at  all  levels  will  eventually  find  on  their  on-line  menus  are  questions  that 
remain  to  be  answered.  Both  these  issues  and  the  related  technologies  are  explored  in  what  follows. 


The  National  Information  Infrastructure  and  New  Opportunities 
For  Distance  Learning 


Much  has  happened  since  Stewart  Brand  (1989)  predicted  the  inexorable  merging  of  the  publishing, 
broadcasting  and  computer  industries.  Private  competition  among  cable,  telcos  and  the  computer  industry, 
aided  by  government  legislation  and  programs,  have  made  Brand's  words  seem  prophetic.  A  world  in  which 
analog  information  originates  from  and  is  controlled  by  a  central  source  and  broadcast  at  fixed  times  to 
many  simultaneously  is  being  replaced  by  digital  information  originating  from  a  myriad  of  independent 
sources  and  available  asynchronously.  The  capture,  manipulation,  transmission,  and  consumption  of 
information  in  digital  form  has  become  a  critical  function  of  our  economy  (Shepard,  1994).  Exponential 
advances  in  microchip  technology  and  networked  computers  (Gilder,  1994a)  have  resulted  in  wide-spread 
application  of  peer  to  peer  computing  .  Increased  "bandwidth"  and  advances  in  compression/decompression 
technology  are  enablinging  the  seamless  integration  of  multimedia  (voice,  data,  graphics  and  video)  over 
local  and  wide  area  networks.  The  multimedia  content  carried  on  these  networks  touches  upon  all  aspects  of 
life:  education,  entertainment,  community,  commerce,  and  health. 


An  Interconnected  World 

Today's  networks  number  more  than  100,000  and  represent  a  panoply  of  private,  public  and 
government  local  (LAN),  metropolitan  (MAN)  and  wide  area  networks  (WAN).  Among  them,  the  Internet, 
a  world-wide  "network  of  networks"  (Krol,  1994)  that  share  TCP/IP  and  similar  protocols  is  the  most 
widely  known  and  used.  The  Internet  has  become  popularized  by  the  media  and  is  sometimes  referred  to  as 
"cyberspace,"  a  concept  first  described  in  popular  Canadian  science  writer  William  Gibson's  1984  novel 
Neuromancer.  The  Internet  has  grown  exponentially  at  a  rate  of  ten  percent  a  month  and,  by  1994,  exceeded 
two  million  interconnected  computers  worldwide  (Comer,  1995).  The  approximately  thirty  million  users  of 
the  Internet  vary  from  individuals  with  local  area  network  access  to  individuals  who  subscribe  to  service 
providers  such  as  America  Online,  Compuserv  or  Prodigy  for  access  from  computer  modems.  This  trend 
toward  computer-mediated  communication  (CMC)  over  long  distances  is  further  accelerated  by  the 
commercial  development  and  marketing  of  personal  digital  assistants  (PDAs)  and  the  newly  emerging  array 
of  personal  communications  services  (PCS)  that  add  digital  cellular  telephones  and  wireless  digital  networks 
to  the  mix. 

Private  and  government  efforts  to  design,  test  and  build  the  electronic  infrastructure  to  succeed  the 
Internet-the  "National  Information  Infrastructure"  (Nil),  or  so-called  "Information  Superhighway  "-have 
taker  on  a  new  urgency  over  the  last  several  years.  This  urgency  is  driven  both  by  competition  among 
telcos,  cable,  and  computer  companies  for  a  share  of  the  lucrative  telecommunications  market  and  by  the 
desire  to  apply  technology  in  the  service  of  society.  While  rooted  in  different  motivations  and  goals, 
private  and  government  initiatives  have  complemented  each  other  and  have  hastened  the  development  of  an 
information-based  society  and  economy.  Indeed,  Peter  Drucker  (1994)  has  noted  the  emergence  of 
"knowledge  workers"  as  a  dominant  group  in  our  society  and  predicts  that  by  the  end  of  the  twentieth 
century  they  will  constitute  a  third  or  more  of  the  United  States  workforce.  Knowledge  workers  require 
extensive  formal  education  and  the  ability  to  acquire  and  apply  theoretical  and  analytical  knowledge.  Like 
others,  they  will  have  to  acquire  the  habit  of  continuous  learning  to  remain  productive. 

Technology  does  not  develop  in  a  vacuum.  Whether  it  is  "pushing"  society  along  or  being 
"pulled"  by  societal  forces,  any  examination  of  its  effects  on  learning  must  first  acknowledge  the  larger 
changes  taking  place  in  society  to  ensure  that  technological  solutions  are  applied  appropriately.  The 
emerging  global  economy  presents  several  distinct  problems:  (1 )  a  mobile,  dispersed  workforce;  (2) 


displaced  unskilled  workers;  and  (3)  an  information-based  economy  requiring  highly  skilled  (knowledge) 
workers  .  A  highly  educated,  flexible  workforce  will  be  a  necessity  for  the  United  States  to  remain 
competitive  in  the  twenty-first  century.  It  is  clear  that  an  educated  peison  will  be  someone  who  has  "learned 
how  to  learn"  and  who  continues  to  learn  throughout  his  or  her  lifetime  (Drucker,  1994).  Varied  resources, 
innovative  educatioml/training  strategies  and  technologies  foremost  among  them,  will  be  required  to  meet 
these  challenges. 


Non -Traditional  Learning  Environments 

The  challenges  inherent  in  the  global  economy  affect  all  stages  of  education  from  the  formative 
childhood  years  through  college/university  schooling  to  continuing  professional  and  non-professional 
education.  The  notion  of  education  as  a  formal  process  occurring  at  a  specified  location  during  a  fixed  time- 
frame is  not  adequate  to  meet  the  requirements  of  the  twenty-first  century.  A  paradigm  shift  in  the 
operational  concept  of  the  teaching/learning  process  is  a  necessary  prerequisite  to  adequately  addressing 
future  challenges.  This  is  not  to  propose  a  zero  sum  game:  abandon  traditional  education  and  embrace  a 
world  of  virtual  education.  Instead,  the  paradigm  suggested  is  one  that  integrates  a  new  role  for  teacher  and 
learner  within  traditional  education  and  looks  to  identify  transition  areas  where  newer  technologies  can 
provide  more  effective  learning  environments. 

Newly  available  tools  for  storing,  accessing,  creating,  delivering,  displaying  and  evaluating 
information  are  transforming  the  learning  process.  As  networked  information  resources  become  more  widely 
accessible,  learning  can  occur  anytime,  anywhere  (Twigg,  1994).  When  access  to  networked  information  is 
available  in  schools  and  integrated  into  curricula,  several  aspects  of  the  learning  environment  are 
transformed.  Most  notably,  students  become  active  participants  in  learning  rather  than  passive  recipients, 
and  teachers  embrace  a  more  active  role  in  facilitating  learning.  Students  assume  more  individual 
responsibility  for  achieving  learning  outcomes  and  explore  alternative  sources  of  information  and 
knowledge.  Students  develop  an  understanding  of  how  to  use  various  tools  to  search  and  retrieve  networked 
information.  Additionally,  they  learn  to  manipulate  information  and  to  analyze  and  work  with  symbols. 
Teachers  collaborate  with  networked  peers  and  become  active  mentors,  group  project  leaders  and  designers  of 
learning  experiences.  Networked  teams  of  teachers  and  students  collaborate  to  apply  interactive  high 
performance  technology  to  solve  real-world  problems,  to  retrieve  information  from  electronic  libraries,  and 
perform  scientific  experiments  in  simulated  environments. 

Effectiveness  studies  of  networked  technologies  in  schools  have  demonstrated  increased  student- 
student  and  student-teacher  interaction  and  increased  student-teacher  interaction  with  lower  performing 
students  without  decreasing  the  use  of  more  traditional  forms  of  communication  (Interactive,  1993; 
D'Souza,  1993)  demonstrated  that  students  who  conscientiously  use  personal  electronic  mail  as  an  additional 
means  of  self-reflection  develop  greater  conceptual  understanding  of  the  subject.  Additionally,  hypermedia 
environments  have  been  shown  to  afford  students  more  insight  into  organizing  and  synthesizing 
information  (Turner  and  Dipinto,  1992) 

The  accessibility  of  multimedia  over  networks  is  expected  to  extend  the  range  of  effectiveness  of 
distance  learning  applications  (Collis,  1991;  Steinberg,  1992;  Kaye,  1992).  Furthermore,  networks 
encourage:  (1)  cooperative  distance  learning  among  peer  groups  of  remotely  connected  students;  (2)  an 
organizational  framework  for  mentoring,  apprenticeships,  peer  collaborations  and  (3)  direct  participation  in 
scientific  research  (Civille,  Fidelman,  and  Altobello,  1993).  Ideas  such  as  ubiquitous  computing  (Weiser, 
1991)  and  the  design  of  computer-supported  cooperative  work  (Ishii,  Kobayashi,  and  Arita,  1994)  may 
provide  the  necessary  ingredients  to  create  a  seamless  web  of  media  (text,  video,  audio,  graphics,  video-on 
demand  and  video  conferencing)  as  part  of  a  broadband  National  Information  Infrastructure. 

Networked  applications  reach  beyond  the  schools  and  extend  into  the  community  of  work  where 
the  integration  of  hardware  and  software  connect  workers  with  a  vast  array  of  information,  training  and 
employment  services.  Much  needs  to  be  accomplished  to  expand  these  services  beyond  their  nascent  stages 
into  a  seamless  web  of  educational  and  training  opportunities  available  during  formative  schooling  and 
continuing  into  an  adult  cycle  of  productive  "life-long  learning". 


634 


National  Information  Infrastructure  for  Life-Long  Learning 

The  creation  of  the  National  Information  Infrastructure  (Nil)  will  effect  many  aspects  of  society: 
electronic  commerce,  agile  manufacturing,  government  services,  civic  networking  and  life-long  learning. 
The  intent  is  to  "inteprate  hardware  and  software  skills  to  make  it  easy  and  affordable  to  connect  people  with 
each  other,  with  computers,  and  with  a  vast  array  of  services  and  information  resources"  (Civille,  Fidelman, 
Altobello,  1993,  p.  1).  The  remainder  of  this  paper  focuses  on  the  availability,  access,  physical  structure 
and  processes  of  the  Nil  relevant  to  applying  distance  learning  concepts  and  technologies.  It  examines  the 
generic  issues  of  distance  learning  within  the  evolving  framework  of  the  Nil  with  some  specific  examples 
of  distance  learning  in  schools,  higher  education  and  the  health  professions. 

Universal  Availability  and  Access 

Without  a  public  infrastructure  that  enables  both  the  public  and  private  worlds  to  expand  and  grow 
with  the  new  technology,  little  real  progress  can  be  made  in  achieving  a  truly  ubiquitous  information 
society  (Heldman,  1994).  A  framework  of  cooperation  between  government,  academia,  industry  and  key  user 
groups  is  necessary  to  meet  urgent  societal  needs,  encourage  research  and  innovation,  and  induce  investment 
from  the  private  sector  (Weingarten,  1993). 

Mason  (1994)  describes  digital  convergence  as  the  most  explosive  technology  confronting  our 
community  today.  Mason's  prescription  for  universal  access  to  the  ubiquitous,  networked,  multimedia 
promised  by  the  Nil  has  three  basic  components.  According  to  Mason  every  community  member  should 
have:  (1)  a  basic  "kit"  of  informatics  equipment  for  display,  input  and  memory;  (2)  unrestricted  access  to 
the  "public"  information  that  is  available  and  (3)  basic  usability  or  friendly  interfaces  to  information. 

Kapor  and  Weitzner  (Kapor,  1993;  Kaporand  V/eitzner,  1993)  present  an  argument  for  the  re- 
wiring of  America  that  results  in  Jeffersonian  networks  promoting  the  openness,  freedom  and  diversity  that 
is  the  true  promise  of  the  technology.  Their  Jeffersonian  ideal  consist  of:  (1)  increased  bandwidth  for  all 
information  sources;  (2)  open  architecture/open  standards;  and  (3)  a  model  of  control  patterned  on  the 
Internet  rather  than  on  the  broadcast  model.  Openness  and  cooperation  between  government  and  private 
enterprise  are  the  crucial  ingredients  to  ensure  universal  availability  and  access  to  the  NH. 

The  federal  government's  role  is  properly  perceived  as  that  of:  (1)  facilitating  private  sector 
investment  in  infrastructure;  (2)  removing  regulatory  barriers,  establishing  standards,  supporting  research, 
prototyping  and  evaluating  "benchmark"  applications  and  (3)  providing  assistance  to  education  and  training 
communities  (Gore,  1994).  Government  also  properly  communicates  a  vision  for  the  education  and  life- 
long learning  applications  on  the  Nil.  State  and  local  governments  would  be  responsible  for  a  large  share  of 
the  investment  in  elementary,  secondary  and  higher  education.  They  are  also  in  a  position  to  remove 
regulatory  and  tariff  barriers  to  Nil  access  in  local  communities. 

The  private  sector  on  the  other  hand  has  responsibility  to:  (1)  build  the  telecommunications 
infrastructure  and  (2)  provide  the  "lion's  share  of  the  nation's  investment  in  applications  for  education  and 
life-long  learning.  And  finally,  the  private  nonprofit  groups  remain  as  providers  of  information  and 
technical  assistance  in  such  activities  as  "Free  Nets"  (best  exemplified  by  the  Cleveland  Free  Net). 

Universal  availability  and  access  for  all  citizens  is  only  ensured  through  an  open  platform  concept 
(Electronic  Frontier  Foundation,  gopher.harvard.edu,  1994).  The  Electronic  Frontier  Foundation  defines  an 
open  platform  as  ranging  from  narrowband  to  broadband  digitally  switched  services  made  available  by  a 
variety  of  providers  over  interconnected  networks  using  a  variety  of  transmission  media:  copper  wire, 
coaxial  cable,  fiber  optics  and  wireless.  Criteria  for  an  open  platform  include:  (1)  widely  available  switched 
digital  connections  at  affordable  prices;  (2)  open  access  to  all  without  discrimination  as  to  the  content  of 
message;  (3)  sufficient  "upstream"  capacity  to  enable  users  to  originate,  as  well  as  receive  good  quality 
video  and  multimedia  services.  This  last  requirement  cannot  be  overemphasized.  As  Balabanian  (1994) 
points  out  in  Computer  Networks  and  Equity,  the  concept  of  access  equity  is  broader  than  just  equal  access 
and  must  apply  equally  to  the  transmission  of  information. 
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Telecommunications 


The  existing  telecommunications  infrastructure  consists  of  telephone,  broadcast,  satellite,  cable  and 
electronic  networks.  Education,  training,  and  life-long  learning  applications  consist  mainly  of  (1)  video 
instruction  (one-way  video  and  two-way  audio  and  two-way  video/audio);  (2)  information  retrieval  from 
databases/libraries;  (3)  two-way  asynchronous  applications  such  as  email,  listservs,  bulletin  boards, 
newsgroups;  (4)  electronic  transfer  of  instructional  software  and  simulations  and  (5)  distance  learning. 

In  the  aggregate,  these  applications  are  representative  of  what  Romiszowski  (1993)  identifies  as 
the  fourth  phase  in  the  development  of  distance  education.  One  that  is  characterized  by  the  use  of  advance 
telecommunications  and  computer-mediated-communication  (CMC).  The  prior  three  phases  were  identified 
as:  (1)  a  print-based  model  of  correspondence  education;  (2)  radio/television  open  broadcast  with 
correspondence  and  print  materials  and  (3)  a  variety  of  teleconferencing  systems.  Taylor  (1992)  and  Miller 
(1992)  identify  similar  phases  in  the  development  of  distance  education.  They  describe  them  as  the 
correspondence  model,  the  multimedia  model  and  the  telelearning  model.  Together  these  authors  foresee  a 
fourth  phase  of  distance  education  dominated  by  computer-mediated-communication  and  advance 
technologies  inexorably  tied  to  the  major  events  that  are  currently  unfolding  in  the  telecommunications 
marketplace. 

The  creation  of  the  Nil  is  driven  by  the  convergence  of  hardware,  software,  networking 
technologies  and  the  quest  for  digital  data.  The  data  highway's  backbone  will  use  every  wide  area 
communication  technology  known:  fiber  optics,  satellites  and  microwaves  (Reinhart,  1994).  On-and  off- 
ramps  connecting  users  to  the  backbone  will  consist  of  fiber,  coaxial  cable,  copper  and  wireless 
technologies.  Network  clients  will  be  served  by  supercomputers,  mainframes,  minicomputers, 
microcomputers  and  massively  parallel  machines.  Software  interfaces  to  the  network  will  consist  of  a  new 
generation  of  "middleware"  known  as  intelligent  agents  or  knowbots  (Roesler  &  Hawkins,  1 994).  Users 
will  receive  and  send  information  on  a  variety  of  devices:  information  appliances,  smart  phones,  personal 
digital  assistants,  set-top  boxes/TVs  and  microcomputers. 

According  to  Reinhart  (1994)  and  Heldman  (1994)  protocols  and  bandwidth  are  the  two  main 
unresolved  technical  issues  relevant  to  the  data  highway's  architecture.  Advanced  protocols  and  increased 
bandwidth  are  crucial  to  establishing  the  full  range  of  voice,  data,  text,  images,  high  resolution  video  and 
interactive  communications  that  are  envisioned  for  the  NIL 


Major  Components:  Broadcast,  Cable,  Telcos,  Networks 

The  technology  components  that  will  eventually  constitute  the  Nil  currently  exist  as  a  disparate 
collection  of  products  and  services.  These  components  are  comprised  primarily  of  broadcast  services,  cable 
companies,  telcos  (the  seven  regional  Bell  companies  (RBOCs)  and  long  distance  companies)  and  various 
interconnected  public  and  private  electronic  networks.  Each  component  pursues  its  own  commercial  or 
public  objectives  and  in  most  instances  has  its  own  communication  protocols,  information  formats,  and 
transmission  capacity  or  bandwidth.  Out  of  this  varied  mix  of  services  and  products  must  emerge  a 
ubiquitous,  affordable,  easy  to  >.'se,  multipurpose,  information  rich  and  open  Nil. 

The  broadcast  and  cable  companies  represent  an  analog  world  of  broadcast  services  with  a  point  to 
multi-point  topology.  Together  they  provide  a  "gateway  model"  where  information  originates  and  is 
controlled  at  a  central  point  and  is  broadcast  simultaneously  to  multiple  locations.  Cable  companies 
presently  have  limited  capacity  for  "point"  to  "point"  or  two-way  communication.  Cable's  strong  suite  is 
their  installed  base  of  broadband  coaxial  cable  that  passes  ninety-seven  percent  of  United  States  households 
(Cable  Television  Laboratories,  1994).  Enhancements  such  as  digital  compression  and  "fiber  to  the  curb  or 
pedestal"  (extending  fiber  beyond  main  trunk  lines  out  into  the  neighborhoods)  are  transforming  cable 
systems  into  high  capacity  digital  transmission  networks.  The  hybrid  fiber  and  coaxial  cable  combination 
provides  cable  companies  with  the  potential  of  providing  interactive  information  services  beyond  the  much 
"hyped"  movies-on-demand.  An  example  of  this  potential  is  seen  in  the  recently  announced  joint  venture 
between  Tele-Communications,  Inc.  and  Microsoft  to  deliver  on-line  services  to  personal  computers  via 
cable  TV  lines  (Farhi,  1994). 
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Electric  utilities  also  provide  a  potential  source  for  a  hybrid  network  of  fiber  and  coaxial  cable. 
Demand-side  Management  (DSM)  is  a  plan  for  utilities  to  share  a  new  distribution  system  with  telephone 
and  cable-TV  companies  that  could  carry  two-way  data,  voice  and  video  on  a  single  line  (Brunet  1995).  The 
plan  ill!  ludes  two  components:  (1)  a  hybrid  network  infrastructure  of  optical-fiber  from  the  electrical  utility 
office  to  a  feeder  location  plus  two-way  coaxial  cable  from  the  feeder  to  residence  and/or  office;  and,  (2)  a  set 
of  microprocessor-based  devices-a  gateway  at  the  utility  substation  and  intelligent  interfaces  at  the  residence 
or  business.  The  DMS  system  would  include  multiple  subsystems:  one  each  for  phone,  data,  and  video 
services. 

Telephone  companies  represent  the  world's  laigest  circuit-switched  distributed  network  designed  for 
point-to-point  communications.  The  Public  Switched  Telephone  Network  (PSTN)  is  a  combination  of 
technologies:  fiber  optics,  copper  wiring,  satellite  and  microwave.  Long  distance  carriers  are  responsible  for 
the  high  speed  lines  interconnecting  regional  and  national  phone  systems  and  for  the  Internet  backbone. 

The  telephone  network  is  being  replaced  with  digital  technology  from  the  network  on  down  to  the 
customer  premises.  Two  major  issues  under  consideration  will  influence  the  development  of  the  Nil:  (1) 
deployment  of  optical  fiber  to  the  customer  premises  at  a  cost  of  billions  of  dollars  and  (2)  what  services 
(such  as  video)  in  addition  to  voice  will  be  allowed  over  the  switched  telephone  network.  Competition 
abounds  among  RBOCs,  long  distance  providers  and  cable  companies  over  the  issues  of  local  versus  long 
distance  services,  value  added  services  and  delivery  of  video  service  to  the  home.  Federal  legislation  will  be 
required  to  resolve  these  competitive  issues  and  most  likely  will  result  in  more  open  competition  among 
the  major  players. 

In  the  minds  of  many,  today's  Internet  stands  as  the  model  network  to  be  emulated  in  the 
development  of  the  National  Information  Infrastructure.  The  Internet  represents  an  open  system  for 
communication  where  information  has  multiple  points  of  origin  and  is  sent  over  a  distributed  network  of 
interconnected  computers.  It  is  perceived  as  a  model  because  it  stands  in  stark  contrast  to  the  concept  of  hub 
and  spoke  networks  where  information  and  services  originate  and  are  delivered  from  a  central  point. 

The  Internet  is  a  highly  successful  and  well  regarded  example  of  how  multiple  packet  switching 
networks  can  be  interconnected  to  form  a  network  of  networks.  It  is  referred  to  as  a  network  of  networks 
because  it  is  made  up  of  networks  interconnected  by  routers.  Sometimes  it  is  referred  to  as  a  "virtual 
network"  because  it  presents  the  illusion  of  a  single  large  network.  This  configuration  has  led  to  much 
success  since  the  ability  to  connect  multiple  types  of  networks  allows  each  group  in  an  organization  to 
choose  a  network  technology  that  best  suits  the  group's  needs  and  budget  (Comer,  1995).  Its  success  is 
further  exemplified  in  its  extension  into  the  National  Research  and  Education  Network  (NREN)  which  has 
become  a  catalyst  for  the  development  of  the  high  speed  communications  and  information  systems  that  will 
become  the  Nil. 

Software  applications  on  the  Internet  use  the  same  general  structure  know  as  client  server 
computing.  The  key  communication  protocol  used  on  the  Internet  is  called  the  Internet  Protocol  (IP).  All 
computers  that  connect  to  the  Internet  must  follow  the  rules  of  IP.  Transmission  Control  Protocol  or  TCP 
is  the  companion  software  that  provides  reliable  error-free  communication.  Together  this  communication 
protocol  is  known  as  TCP/IP.  The  National  Science  Foundation  formed  the  backbone  of  the  In^rnet  up 
until  1992.  Since  1992,  a  non-profit  company  called  Advanced  Networks  and  Services  (ANS)  comprised  of 
IBM,  MERIT,  and  MCI,  has  provided  the  high  speed  backbone  for  the  Internet  (Comer,  1995).  ANS  owns 
the  transmission  lines  and  computers  that  comprise  the  network.  This  event  is  seen  by  some  as  the  first 
step  toward  the  commercialization  and  privatization  of  the  Internet. 


Evolving  Physical  Infrastructure 

How  will  these  technology  components  and  their  attributes  evolve  into  the  National  Information 
Infrastructure?  Ccrf  (1991),  Dertouzos  (1991),  Heldman  (1994),  Negroponte  (1991)  and  Tesler  (1991)  offer 
some  prudent  observations  in  their  writings  on  the  subject  of  networked  communications.  Several 
technologies,  some  new,  some  not  so  new,  appear  in  their  writing.  Integrated  Digital  Services  Network 
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(ISDN),  Asynchronous  Transfer  Mode  (ATM),  and  Synchronous  Optical  Network  (SONET)  are  among  the 
most  frequently  mentioned  technologies  that  impact  on  the  development  of  future  electronic  networks. 
Satellite  and  wireless  technologies  are  also  perceived  as  a  salient  component  of  the  overall  architecture  of 
the  NIL 

ISDN  was  conceived  by  the  International  Telegraph  and  Telephone  Consultative  Committee 
(CCITT)  in  1971  as  a  scheme  to  replace  the  plain  old  telephone  system  (POTS)  with  an  all  digital  network 
that  integrated  voice  and  data  on  a  single  subscriber  line.  ISDN  is  a  network  technology  that  provides  a  way 
of  utilizing  narrowband  technology  (N-ISDN)  that  enhances  the  installed  base  of  copper  wires  for  shared 
voice/data  in  a  digital  mode  (64kbps- 128kbps)  and  allows  for  phase  in  of  broadband  technology  (B-ISDN)  in 
the  early  stages  of  fiber  development  (Kaman,1993).  Basic  Rate  ISDN  offers  speed  of  more  than  100 
kilobits  per  second  over  residential  copper  wire  while  Primary  Rate  ISDN  offers  more  than  a  megabit  per 
second  using  specialized  copper  access  plant.  The  implementation  of  ISDN  would  maximize  the  use  of 
existing  copper  plant  requiring  no  change  in  customer  premises  wiring  or  equipment  during  the  transition 
phases  into  broadband  fiber  optic  technologies  necessary  to  achieve  the  goals  of  the  Nil. 

ISDN  offers  a  framework  within  which  existing  analog  networks  can  evolve  to  meet  changing 
users  needs  for  higher  capacity  broadband  services  (Heldman,  1994).  Broadband-ISDN  (B-ISDN)  would 
replace  the  physical  infrastructure  of  the  public  switched  telephone  network  with  optical  fiber  cable  with 
transmission  rates  of  !55  megabits  to  2.4  gigabits  per  second.  It  would  also  utilize  fast-packet  switched 
protocols  and  make  use  of  virtual  channels  that  do  not  require  a  pre-specified  bit  rate.  This  ability  to  provide 
transport  of  information  independent  of  its  service  type  is  referred  to  as  band-width-on  demand.  Heldman 
(1994)  also  points  out  the  importance  of  ISDN  standards  as  numerous  vendors  attempt  to  understand  and 
standardize  various  interfaces  for  the  exchange  of  integrated  voice,  data,  and  video  information  in  the  form  of 
bursty,  fast  packet,  circuit  and  channel  type  transmission  and  switching.  While  not  the  definitive  answer  to 
achieving  the  goal  of  universal  information  services,  ISDN  offers  a  phased  plan  for  proceeding  into  the 
future. 

Asymmetric  digital  subscriber  loop  (ADSL)  is  another  technology  that  phases  in  digital 
transmission  over  copper  wires.  ADSL  permits  transmission  of  a  single  compressed,  high  quality  video 
signal  (VCR  quality)  at  a  rate  of  1.5  megabits  per  second  in  addition  to  ordinary  voice  phc  le  conversations 
(Kapor,1993). 

The  framework  for  future  high  speed  wide  area  digital  networks  is  the  photonic  standard  called 
Synchronous  Optical  Network  or  SONET.  SONET  supports  a  multiplexed  hierarchy  of  transmission  speeds 
of  51  megabits  to  2,400  megabits  per  second  and  allows  for  multiplexing  and  demultiplexing  varying  data 
speeds  without  breaking  down  each  stream  into  individual  components  (Cerf,  1991).  Photonic  switching 
research  promises  optical  switching  machines  with  a  terabit  or  1  trillion  bits  per  second.  Such  capacity  will 
be  necessary  in  the  future  for  transmission  and  retrieval  of  massive  amounts  of  high  resolution  video  and 
three-dimensional  images. 

Asynchronous  Transfer  Mode  (ATM)  is  fast  becoming  a  world  wide  digital  communication 
standard.  ATM  is  poised  to  become  the  communication  technology  protocol  of  the  future  (Heldman,  1994). 
Its  strengths  are  flexibility  and  speed.  ATM  can  handle  real-time  traffic  like  voice  and  video  and  is  also 
effective  for  more  traditional  data  communication.  ATMs  fixed  length  53  byte  cells  allows  the  integration 
of  data  media  operating  at  different  rates.  This  predictability  allows  scalable  network  performance.  Although 
it  can  run  on  just  about  any  medium,  ATM  was  developed  with  the  intention  to  run  on  fiber-optic  networks 
as  defined  by  SONET  a  US  standard  and  (SDH)  synchronous  digital  hierarchy,  a  European  standard. 

ATM  is  the  first  technology  to  provide  a  common  format  (cell)  for  bursts  of  high  speed  data  and 
the  ebb  and  flow  of  the  typical  phone  call.  ATM  is  scalable  and  equally  at  home  in  any  network,  local  or 
wide,  public  or  private  (Heldman,  1994).  Given  its  attributes,  ATM  could  erase  *he  distinction  between  local 
area  networks  (LANS)  and  wide  area  networks  (WANS)  to  provide  the  widely  sought  after  goal  of  seamless 
networking. 

Wireless  networking,  which  permits  continuous  access  to  the  services  and  resources  of  land-based 
nei  works  (Forman  and  Zahorjan,  1994),  also  will  be  an  important  technology  component  of  the 
infrastructure.  Communication  on  wireless  networks  is  via  modulating  radio  waves  or  pulsing  infrared 


638 


light.  Wireless  communication  is  linked  to  the  wired  network  infrastructure  by  stationary  transceivers.  This 
mobile  computing  capability  provides  a  technology  that  enables  access  to  digital  resources  at  any  time  from 
any  location. 

Although  growing  rapidly~to  the  tune  of  sixteen  billion  dollars  in  licensing  fees  to  the  FCC  for 
personal  communication  services  (PCS)  bandwidths  (Arnst,  1995),  wireless  technology  is  not  without 
problems.  It  is  characterized  by  low  bandwidth  and  is  subject  to  high  error  rates  and  surrounding 
environmental  interference.  Current  R&D  efforts  supported  by  telecommunications  companies  are  aimed  at 
solving  these  problems.  The  recent  promising  development  of  code  division  multiple  access  (CDMA) 
allows  wireless  systems  to  have  ten  times  the  capacity  of  old  wireless  systems.  By  limiting  the  time-and- 
place  restrictions  inhcent  in  desktop  computers  and  wired  networks,  the  potential  of  mobile  computing 
increases  user  accessibility  to  network  services  around  the  world. 

Satellites,  Instructional  Television  Fixed  Services  (ITFS)  and  microwave  technologies  currently 
provide  alternative  delivery  systems  for  instructional  programming  to  the  public  schools  (Lane,  1993). 
Although  primarily  analog  in  nature  these  technologies  will  be  integrated  into  the  emerging  digital 
information  infrastructure.  Several  factors  portend  this  integration. 

Geosynchronous  satellites  (those  in  an  orbit  equivalent  to  the  earth's  rotation)  provide  "store  and 
forward"  applications  such  as  computer  conferencing  and  voice  mail.  Additional  applications  will  be 
forthcoming  from  Motorola  Inc.'s  (Iridium)  and  Comsat's  global  satellite  communications  programs. 
Recently,  Teledesic  Corp  (a  Microsoft  and  McCaw  Cellular  Communications  Inc.  start-up  company) 
announced  its  intentions  to  provide  a  network  of  840  satellites  to  bring  telecommunications  to  remote  areas 
around  the  world  by  the  year  2001  (Sugawara,  1994).  Teledesic  embraces  the  technology  developed  under  the 
Strategic  Defense  Initiative's  "brilliant  pebbles"  program.  Other  developments  such  as  Direct  Broadcast 
Satellite  (DBS)  and  the  recently  announced  Digital  Satellite  Services  (DSS)  offer  digital  programming  to 
small  receiver  dishes  (ranging  from  three  feet  to  eighteen  inches  in  diameter)  conveniently  located  at 
consumer's  homes.  As  these  services  penetrate  the  home  market  it  becomes  increasingly  clear  that  they  will 
inevitably  merge  with  the  Nil. 

The  technology  blueprint  for  the  future  will  involve  the  application  of  common  standards  and 
communication  protocols  to  interconnect  local  and  wide  area  networks  linking  people  together  over  a 
National  Information  Infrastructure.  Open  Systems  Interconnection  (OSI)  seven  layer  architectural  reference 
model  and  standard  for  communications  among  computers  and  terminals  will  provide  the  evolving  guide  for 
this  blueprint.  Fast-packet  architectures  will  be  the  key  to  this  development  and  will  be  part  of  both  public 
and  private  networks.  The  resulting  integrated  networked  environment  will  appear  as  a  "virtual"  network  for 
the  end  user,  broadening  communication,  giving  users  access  to  varied  applications  and  hopefully  reducing 
overall  costs  (McGee,  1994). 

Asynchronous  Transfer  Mode  (ATM)  appears  to  be  the  protocol  of  choice  for  full  multimedia 
capabilities.  McGee  (1994)  predicts  that  the  focus  of  the  networking  industry  for  the  rest  of  this  decade  will 
be  dominated  by  the 

integration  and  evolution  of  today's  multiple,  specialized  technologies  into  a  fewer  number  of  higher  speed, 
multidimensional  technologies. 

The  litmus  test  of  the  technologies  that  represent  the  information  infrastructure  ultimately,  will 
not  be  judged  by  bandwidth  capacity  or  number  of  executable  operations  per  second  but  rather  by  the 
character  of  its  content.  The  remainder  of  this  paper  explores  potential  content  in  the  civic  and  education 
arena,  with  special  emphasis  on  distance  learning. 

Distance  Learning  Opportunities 

Community  Services  Networks 

Many  opportunities  for  networked  community  services  are  outlined  in  A  National  Strategy  fur 
Civic  Networking  :  A  Vision  of  Change  (Civille,  Fidelnian,  &  Altobello,  October  1993).  Their  review  is 
replete  with  examples  of  networked  applications  that  cut  across  all  aspects  of  civic,  education,  health,  work, 


and  community  activities.  The  projects  reviewed  are  exemplary  of  the  salutary  and  empowering  effects  of  a 
national  strategy  for  civic  networking  predicated  on  a  National  Information  Infrastructure.  They  illustrate 
how  integrated  networks  are  able  to  provide  endless  opportunities  for  resource  sharing  and  collaboration 
among  individuals  and  communities.  Two  of  the  most  prominently  mentioned  projects  are:  the  Blacksburg 
Electronic  Village  and  Smart  Valley. 

Blacksburg  Electronic  Village  in  southwestern  Virginia,  is  a  testbed  for  the  delivery  of  business, 
community,  education,  health,  and  library  services  over  narrow  and  broadband  ISDN  networks.  This  project 
attempts  to  enhance  the  quality  of  people's  lives  by  electronically  linking  the  residents  of  the  community  to 
each  other  and  to  worldwide  networks.  Blacksburg  has  become  a  laboratory  to  develop  a  prototype  for  the 
concept  of  a  National  Information  Infrastructure.  A  hallmark  of  this  experiment  is  its  emphasis  on 
interactions  between  people  rather  than  focusing  on  specific  technologies.  A  Local  Research  and  Education 
Network  (LREN)  is  envisioned  for  educational  uses.  This  local  network  would  connect  to  the  developing 
National  Research  and  Education  Network  for  access  to  database  and  increased  access  to  other  students, 
teachers  and  researchers.  The  community  of  Smart  Valley,  in  Palo  Alto,  California,  is  also  envisioned  to 
develop  an  electronic  community  that  will  benefit  all  sectors  of  the  community:  education,  healthcare  local 
government,  business  and  home  applications. 

An  important  ingredient  of  both  these  projects  is  the  "bottom  up"  approach  to  designing  the 
testbed  networks.  Instead  of  a  "top  down"  systems  rule  approach,  the  individual  needs  and  desires  of 
community  members  are  guiding  the  development  of  the  integrated  networks  and  services.  As  Glider  (1994) 
has  pointed  out  networks  should  be  allowed  to  evolve  as  an  organic  process  and  grow  from  needs  rather  than 
be  imposed  from  the  top.  Advances  in  technology  will  be  more  secure,  enduring  and  profitable  if  they  result 
from  careful  examination  of  user  requirements  and  needs. 


Distance  Learning  Initiatives 

Plans  to  use  thi  National  Information  Infrastructure  for  education  and  life-long  learning  address  the 
challenges  of  the  emerging  global  information  economy  identified  in  a  previous  section  of  this  paper. 
Federal,  state  and  local  governments  and  private  business  have  initiated  programs  to  set  the  stage  for 
meeting  these  challenges.  Several  programs  in  the  federal  government  (Aeronautics  &  Space,  Ener 
Federal  Communications  Commission,  Housing  and  Urban  Development,  Interior,  and  the  Office  oi 
Science  and  Technology  Policy)  have  launched  initiatives  (Gore,  1994). 

These  initiatives  involve:  (1)  developing  K-12  computing  and  communication  applications 
supporting  a  new  learning  paradigm;  (2)  the  development  of  digital  libraries  to  utilize  distributed  databases 
around  the  nation  and  the  world;  (3)  training  programs  to  use  the  Nil  to  enhance  the  skills,  education  and 
training  of  the  American  workforce;  (4)  distributing  K-12  curriculum  over  the  Internet;  (5)  regulations  for 
reasonable  rates  for  telecommunications  by  radio,  television,  wire,  satellite  and  cable;  (6)  establishing  a 
high  speed  communications  infrastructure  for  research  and  educational  use  (NREN)  and  (7)  the  development 
of  software,  interfaces,  and  tools  necessary  for  the  educational  use  of  the  NIL 

Current  initiatives  to  develop  the  communication  technologies  and  tools  for  the  "information  age" 
have  the  potential  to  revolutionize  traditional  educational  environments  and  practices.  Traditional 
educational  settings  where  learners  are  passive  recipients  of  information  provided  in  lecture  formats  or  from 
linear  p.inted  materials  are  fast  giving  way  to  computer-mediated-communication  of  multimedia  information 
over  local,  metropolitan  and  wide  area  networks.  Such  environments  change  the  focus  from  teacher  control 
to  learner  control,  and  from  highly  structured  learning  to  flexible  and  responsive  settings  that  offer  varied 
learner  options.  Linear  presentation  gives  way  to  random  access;  passive  attendance  to  active  participation; 
rote  learning  to  developing  problem  solving  competency  and  independent  thinking;  information 
assimilation  to  knowledge  construction;  classroom  facilities  to  authentic  simulated  environments.  This 
transition  from  a  "teaching  environment"  to  a  "learning  environment"  sets  the  stage  upon  which  school- 
based  and  workplace  learners  will  encounter  opportunities  for  life-long  learning. 

These  initiatives,  we  believe,  signify  a  need  to  expand  our  perception  of  distance  learning  to 
encompass  the  opportunity  for  life-long  learning  provided  by  a  variety  of  telecommunication  technologies, 
and  especially  computer-mcdiatcd-communication  (CMC)  over  the  Internet. 
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Indications  that  these  opportunities  are  increasing  are  evident  in  the  public  sector  increase  in  the 
number  and  variety  of  on-line  information  services  and  in  the  tremendous  growth  in  the  multimedia 
software  publishing  business.  The  "big  three"  on-line  services:  Prodigy,  Compuserve  and  America  on-line 
now  claim  more  than  three  million  subscribers  among  them.  It  is  predicted  that  on-line  subscriptions  will 
grow  another  thirty  percent  in  1994  (Eng,  1994).  Sales  of  home  learning  multimedia  software  will  reach 
250  million  in  1^94  and  are  expected  to  reach  one  billion  by  the  end  of  the  decade  (Armstrong  et  al.,  1994). 
Interactivity,  defined  as  active  involvement  and  control  by  participant  or  learner  with  the  entertainment  or 
educational  product  is  a  key  ingredient  to  the  success  of  these  two  industries.  The  merging  of  computers, 
telephones,  satellites,  cable,  television  and  wireless  technologies  into  a  transparent  web  of  networks  making 
a  vast  array  of  information  available  to  homes,  schools  and  businesses  is  another. 

Networked  multimedia  applications  in  the  commercial  and  home  market  are  out  pacing  similar 
ipplications  in  the  schools.  The  good  news  is  that  the  commercial  and  home  market  applications  are  part  of 
the  evolving  information  infrastructure  that  is  beginning  to  penetrate  the  educational  market.  In  1987,  fewer 
than  ten  states  were  investing  in  distance  learning.  By  1988  two-thirds  of  the  states  reported  involvement  in 
distance  learning  projects  (U.S.  Congress,  1989). 

Several  ongoing  programs  exemplify  the  potential  for  educational  applications.  The  Global  Access 
Information  Network  (GAIN)  is  a  project  of  NYSERNET,  a  mid-level  Internet  service  provider  in  New 
York.  The  project  is  supported  by  the  Apple  Library  of  Tomorrow  and  private  foundations.  GAIN  provides 
Internet  services  to  five  public  libraries  and  an  Indian  Nation  school.  In  addition  to  collaboration  between 
local  libraries  and  local  schools,  the  community  has  benefitted  from  access  to  medical  information.  The 
National  Capital  Area  Public  Access  Network  (CapAccess)  in  Washington,  D.C.  works  with  public  school 
librarians  over  a  three  state  area.  (CapAccess)  provides  free  Internet  accounts  to  students  who  use  Internet 
mail  to  subscribe  to  school  topics  of  interest.  In  Montana,  the  Big  Sky  Telegraph  project  links  schools  and 
libraries  enabling  students  to  study  Russian  with  Russian  pen  pals  oversees  and  to  study  chaos  theory  with 
scientists  from  the  Massachusetts  Institute  of  Technology. 

High  Performance  Computing  and  Communications  (HPCC)  programs  have  spawned  some 
exciting  distance  learning  projects  as  part  of  the  InFoMall  concept  (Mills  and  Fox,  1994).  InFoMall  is  a 
program  led  by  the  Northeast  Parallel  Architectures  Center  (NPAC)  to  create  an  innovative  technology 
transfer  program  for  the  "information  age".  It  is  intended  to  address  the  problem  of  HPCC  software 
development  in  the  "real  world"  of  industry,  education  and  society.  The  (NPAC)  InFoMall  is  charged  with 
converting  the  United  States  lead  in  HPCC  technology  into  a  global  economic  competitive  advantage  and 
into  corresponding  benefits  in  health  care,  education  and  society  in  general. 

NPAC  plans  two  demonstration  projects  in  education  through  the  implementation  of  NYNET. 
NYNET  is  a  high-speed,  fiber-optic  communications  network,  based  on  Asynchronous  Transfer  Mode 
(ATM)  technology.  NYNET  will  interconnect  multiple  computing,  communications  and  research  facilities 
in  New  York  State.  The  first  demonstration  project  will  repackage  Grand  Challenge  scientific  simulations 
into  educational  modules  for  school  age  children.  The  modules  will  run  on  high  performance  computers  at 
NPAC  and  will  be  distributed  to  schools  over  the  NYNET  high-speed  network.  Students  and  teachers  will 
become  interactively  engaged  with  the  simulations  and  manipulate  "real  world"  variables.  The  second 
project  is  titled  "The  Interactive  Journey"  and  will  allow  interactive  "real-time"  tours  of  New  York  State 
based  on  LANDSAT  satellite  images.  Students  and  teachers  will  navigate  through  three  dimensional  space 
d  manipulate  archeological,  geological,  environmental,  business  and  demographic  variables. 

Networked  applications  of  distance  learning  are  occurring  with  increasing  frequency  over  the 
Internet.  Paulsen  (1995)  organizes  computer-mediated-communication  (CMC)  according  to  four 
communication  paradigms:  information  retrieval;  electronic  mail;  bulletin  boards  and  computer 
conferencing.  He  further  elaborates  on  this  scheme  by  including  a  dimension  of  interactivity  distinguishing 
among:  one-alone;  one  to  one;  one  to  many  and  many  to  many.  This  scheme  allows  a  matrix  approach  for 
determining  the  interrelationship  between  type  of  communication  and  level  of  interactivity  which  is  a  useful 
tool  for  planning  and  designing  specific  learning  experiences  based  on  intended  outcomes. 

Chris  Dede  (1993)  proposes  a  broader  conceptual  scheme  for  educational  uses  of  the  Nil  that 
includes  knowledge  utilities,  virtual  communities  and  synthetic  environments.  Dede's  framework 
acknowledges  the  links  between  workplace  and  school-based  learning  that  are  necessary  to  address  the 
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transformation  of  work  brought  about  by  the  knowledge-based  economy.  Dede  attempts  to  explore 
interactive  services  that  meet  the  needs  of  both  workplace  and  school-based  learners.  His  framework  relies 
on  collaborative  learning  and  recognizes  the  necessity  for  integrating  interpersonal  with  technology  mediated 
communication. 


Dede's  concept  of  a  "virtual  forum  for  learning"  is  in  response  to  his  perceived  need  for  a  workforce 
trained  in  ad  hoc  or  "just-in-time"  decision  making  and  continual  learning- while-doing  on  the  job.  Dede 
predicts  that  three  types  of  applications  will  come  to  dominate  distance  learning  provided  by  distributed 
desktop  computers  over  wide  area,  high-bandwidth  networks:  (1)  learning  by  assimilation;  (2)  learning  by 
analogizing  and  (3)  learning  by  doing.  Dede's  scheme  capitalizes  on  the  Nil  through  a  "  irtual  forum  for 
learning"  that  leverages  emerging  technological  capabilities  to  promote  learning-on-demand  within  school- 
based  and  the  evolving  workplace  environment  (Dede,  1993,  p.  6). 

Rheingold  (1993),  Savetz  (1994)  and  Weiser  (1994)  promote  the  potential  of  MUDs  (Multi-User 
Dungeons  or  Multi-User  Dialogues  or  Domains)  as  a  distance  learning  application  of  "virtual 
communities".  MUDs  are  typically  computer  programs  that  users  log  into  and  explore  for  recreational 
purposes.  MUDs  have  been  in  existence  for  about  ten  years,  the  first  one  modeled  on  a  Tolkienesque 
domain  of  dwarfs  and  treasure,  warriors  and  wizards,  swordplay  and  magic  knov    us  "the  Land"  (Rheingold 
1993,  p.  155).  Recently,  networked  educational  applications  of  MUDs  have  appeared.  Most  notably,  th^ 
Jupiter  Project,  MediaMOO  and  Virtual  Online  University  (VOV*  Each  of  these  projects  is  described  as  a 
MOO  (a  Multi-User  Dimension  or  object  oriented  environment).  They  are  database  programs  running  on 
servers  that  allow  users  remote  access  to  become  active  learners  in  a  text-based  virtual  reality.  As  such,  they 
provide  a  casual  environment  integrating  virtual  and  real  worlds.  Many  other  applications  of  MUDs  provide 
a  variety  of  electronic  classrooms,  electronic  tutorials  and  seminars. 

Project  Jupiter  is  a  multimedia,  international  MUD  intended  as  a  working  tool  for  designers  of 
virtual  workplaces  of  the  future  (Rheingold,  1993).  MediaMOO  created  in  1993,  is  a  virtual  version  of 
MIT's  Media  Laboratory.  Virtual  Online  University  (VOU)  is  the  first  virtual  online  university  on  the 
Internet  (beta  test  begun  September,  1994)  (R.  Donnelly,  personal  E-mail  communication,  September  7, 
1994).  VOU  is  a  model  for  a  Virtual  Education  Environment  (VEE)  that  combines  the  academic 
environment  of  a  physical  world  university,  the  wide  reach  of  the  Internet  and  the  creative  environment  of  a 
MUD. 

Other  networked  distance  learning  applications  build  upon  the  "peer  to  peer"  distributed  computer 
environment  of  the  Internet.  The  client/server  functionality  of  this  distributed  environment  provides 
individual  access  from  microcomputers  and  workstations  to  a  wide  variety  of  information/education 
resources.  Individuals  access  and  interact  with  these  resources  with  the  four  basic  Internet  services:  e-mail; 
Telnet  (remote  login);  FTP  (file  transfer  protocol)  and  browsers  (Gopher,  Mosaic,  and  others).  A  plethora  of 
resources  are  available  using  these  tools.  Until  recently,  however,  few  educational  applications  were 
available.  Resources  were  largely  information  and  reference  databases  organized  for  access,  retrieval  and 
research  purposes.  This  has  changed  with  the  recent  interest  and  development  of  the  World  Wide  Web 
(WWW). 

The  World  Wide  Web  is  a  hypertext-based  system  for  finding  and  accessing  Internet  resources 
(Krol,  1994).  Use  of  the  World  Wide  Web  accelerated  with  the  development  and  introduction  of  Mosaic,  a 
multimedia  interface  (browser)  to  Internet  Resources.  Web  servers  have  doubled  every  four  months  over  the 
three  years  between  April  1991  and  April  1994  (Berners-Lee  et  al.,  1994)  Mosaic  is  a  hypermedia  tool  thai 
has  the  capacity  for  handling,  along  with  text,  audio,  still  images  and  motion  video.  Mosaic  executes 
multiple  communication  protocols  through  Uniform  Resource  Locators  and  is  platform  independent. 
Because  of  this  functionality  and  its  graphical  user  interface  (GUI),  Mosaic  sparked  an  enormous  interest  in 
creating  World  Wide  Web  sites  (programs  running  on  server  computers)  that  provided  opportunities  for 
multimedia  distance  learning.  The  popularity  and  impact  of  Mosaic  has  spawned  the  creation  of  other  GUI 
browsers  or  clients  for  accessing  Internet  resources. 

It  seems  fair  to  say  thai  the  multimedia  capability  of  the  World  Wide  Web  browsers  was  the 
primary  incentive  motivating  faculty  and  students  to  create  distance  learning  activities  on  the  WWW. 
Suffice  it  to  say,  that  these  distance  learning  opportunities  are  appearing  with  a  frequency  that  makes  it 
difficult  to  systematically  document  and  access  each  application.  For  example  the  authors  have  identified  and 
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documented  numerous  distance  applications  in  the  health  sciences  as  of  November  1994  as  part  of  an 
ongoing  project.  These  applications  have  been  documented  as:  (1)  hypertext  databases  text  only  (2) 
hypertext  databases  text  and  images;  (3)  medical  image  reference  databases;  (4)  case-based  medical  image 
databases  and  (5)  multimedia  databases.  The  levels  of  interactivity  provided  for  the  learner  include  simple 
hypertext  retrieval,  analysis  of  clinical  images,  observation  of  surgical  and  other  medical  procedures 
(QuickTime  and  MPEG  movies)  and  interactive  feedback  (email)  of  patient  case  evaluations.  Anyone  with 
Internet  access  and  a  WWW  graphical  browser  can  review  these  specific  distance  learning  applications  at  the 
following  Web  site  (http://wwwetb.nlm.nih.gov/). 

The  potential  for  networked  collaborative  on-line  distance  learning  can  be  gleaned  from  the  nascent 
MBone  or  "virtual  network"  that  has  been  in  existence  since  early  1992  (Macedonia  and  Brutzman  1994). 
MBone  is  an  acronym  for  Multicast  Backbone.  MBone  originated  from  the  University  of  Southern 
California  Information  Sciences  Institutes  efforts  to  multicast  audio  and  video  from  meetings  of  the 
Engineering  Task  Force.  Multicast  makes  possible  one-to-many  and  many-to-many  network  delivery 
services  for  applications  such  as  videoconferencing  and  audio  where  several  hosts  need  to  communicate 
simultaneously  (Macedonia  and  Brutzman  1994,  p.  30).  Hundreds  of  researchers  currently  use  MBone  to 
develop  protocols  and  applications  for  group  communication. 

Multicasting  has  become  feasible  on  the  Internet  as  the  result  of:  (1)  high  bandwidth  Internet 
backbone  connections  and  (2)  widespread  availability  of  workstations  possessing  adequate  processing  power 
and  built-in  audio  capacity.  Several  distance  learning  applications  have  occurred  using  MBone:  (1 ) 
conferences  on  supercomputing  and  scientific  visualization;  (2)  Radio  Free  VAT;  (3)  Internet  Talk  Radio 
and  (4)  other  remote  learning  activities  such  as  the  Jason  Project  and  the  Naval  Postgraduate  School 
Visualization  Laboratory.  In  practice,  MBone  provides  only  one  to  four  frames  of  video  per  second 
(compared  with  standard  video  of  30  frames  per  second)  accompanied  by  phone  quality  voice.  Nevertheless, 
MBone  may  be  the  precursor  to  the  high  bandwidth  networked  multimedia  applications  promised  by  the 


Opportunities  in  the  Health  Professions 

Several  major  infrastructure  initiatives  are  specifically  relevant  to  health  professions  education:  The 
Integrate-  Advanced  Information  Management  Systems  (IAIMS),  the  multi-agency  High  Performance 
Computing  and  Communications  (HPCC)  programs  and  the  National  Research  and  Education  Network 
(Lindberg,  1993). 

The  IAIMS  is  a  National  Library  of  Medicine  initiative  begun  in  1983  to  promote  the 
development  of  integrated  computerized  communications  systems  across  academic  health  science 
institutions  that  would  link  library  systems  with  individual  and  institutional  databases  for  patient  care, 
research,  education  and  administration.  This  proved  to  be  a  forward  looking  initiative:  IAIMS  participants 
were  the  early  pioneers  of  medical  networking  who  experimented  with  and  integrated  information 
technologies  in  a  medical  environment  that  were  the  precursors  to  current  efforts  to  create  the  Nil. 

Among  the  many  goals  of  HPCC,  the  development  of  the  National  Research  and  Education 
Network  (NREN)  and  the  gigabit  testbeds  for  advanced  networking  are  particularly  pertinent  to  the  issues 
discussed  in  this  paper  (High  Performance  Computing  and  Communications,  1994).  Both  research  and 
education  will  be  advanced  by  efforts  to  connect  health  professional  to  computerized  information  resources 
involving  medical  imaging,  biotechnology  and  information-retrieval  tools.  The  advanced  networking 
projects  support  six  collaborative  testbeds  for  research  on  very  high-speed  networks.  Universities,  national 
laboratories,  supercomputer  centers  and  industry  are  participating  in  this  collaboration.  The  major  goal  of 
this  collaboration  is  to  remove  current  barriers  to  the  efficient  delivery  of  huge  multimedia  files  in  a  shared 
high  performance  computing  environment. 

HPCC  also  supports  projects  in  telemedicine  and  digital  libraries.  As  part  of  this  program  the 
National  Library  of  Medicine  supports  Internet  connection  grants  to  provide  support  for  currently  available 
scr  iccs  over  the  Internet.  The  goal  for  1995  is  to  connect  70  to  100  medical  centers  to  the  Internet.  This 
encourages  baseline  practical  experience  with  current  services  and  hopefully  provides  incentives  for 
involvement  with  the  imminent  advanced  network  technologies. 
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These  initiatives,  coupled  with  the  technological  advances  in  the  commercial  telecommunications 
market  will  have  an  enormous  influence  on:  (1)  the  clinical  practice  of  medicine;  (2)  the  formal  education  of 
health  professionals  and  (3)  the  creation  and  delivery  of  continuing  professional  education. 


Some  Final  Thoughts  on  the  Future  of  Distance  Learning 

School,  home-based  and  workplace  distance  learning  opportunities  will  abound  with  the  advent  of 
the  National  Information  Infrastructure.  If  the  ideal  of  universal  access  (equity  in  access  and  transmission 
and  "common  carriage"  or  subsidized  access  for  those  who  cannot  afford  it)  is  achieved  the  groundwork  will 
have  been  established  for  participation  by  all  citizens.  To  ensure  universal  participation  and  the  achievement 
of  the  distance  learning  goals  projected  for  the  Nil,  four  issues  in  particular  require  attention  :  (1) 
technology's  consequences;  (2)  quality/relevant  content;  (3)  purposefulness  and  (4)  literacy.  Vigilance  for 
technology's  consequences,  both  anticipated  and  unanticipated,  is  foremost  among  them.  Gilder  (1994), 
Norman  (1993),  Papert  (1992),  Postman  (1992),  Zuboff  (1988)  and  others  have  written  eloquently  on  the 
subject  of  technology's  consequences  for  the  design  of  human-machine  interaction,  school,  home  and  work- 
based  activities  and  the  larger  culture.  An  enormous  technological  implementation  such  as  the  Nil  is 
predicted  to  have  far  reaching  consequences  not  all  of  which  are  intended  or  anticipated. 

Individuals  and  institutions  will  have  to  make  accommodations  to  technology's  consequences.  In 
the  area  of  distance  learning  fundamental  questions  will  have  to  be  examined  with  regard  to:  (1)  quality/type 
of  content;  (2  )design/delivery  of  content;  (3)  teacher  role/compensation;  (4)  student  responsibilities;  (5) 
continued  teacher  training;  (6)  role  of  colleges  and  universities/credentials/degree  granting;  (7)  student 
selection/cross  state  and  country  regulations  and  many  more  (Jacobson  1994).  Careful  documentation  and 
assessment  of  the  changes  brought  about  by  these  accommodations  should  be  a  significant  ongoing 
component  of  an  overall  plan  for  the  NIL  Special  emphasis  should  be  placed  on  monitoring  the 
unanticipated  effects  and  planning  for  timely  ;,nd  effective  responses.  Vigilance  for  consequences  will 
increase  the  probability  that  the  emerging  NTI  will  achieve  its  intended  goals. 

Content,  purposefulness,  and  literacy  are  inextricably  bound  togethei  Together  these  three 
variables  will  play  a  significant  role  in  making  universal  access  meaningful  and  ensuring  the  effectiveness 
of  distance  learning  applications.  It  is  not  adequate  simply  for  technology  to  provide  high  tech  conduits  for 
transferring  information.  The  content  traversing  the  conduits  must  be  of  high  quality  and  relevant  to 
enhancing  the  requisite  knowledge  and  skills  of  the  end  user.  The  end  user  in  turn  must  possess  an 
abiding  purpose  for  accessing,  retrieving  and  utilizing  the  information,  and  most  importantly,  fundamental 
literacy  and  information  technology  skills. 

In  the  case  of  distance  learning,  the  issues  and  concerns  relevant  to  content,  purposefulness  and 
literacy  skills  to  a  great  extent  will  determine  the  success  of  the  National  Information  Infrastructure  for  the 
average  citizen. 
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This  paper  was  presented  at  the  meeting  of  the  Association  for  Educational  Communications  and 
Technology  (AECT),  Anaheim  CA,  February  1995.  Portions  of  this  paper  also  appear  in  two  other 
manuscripts  in  press:  B.G.  Wilson  &  P.  Cole,  Cognitive  teaching  models,  in  J.  H.  Jonassen  (Ed.), 
Handbook  of  Research  in  Instructional  Technology,  and  B.  Wilson,  J.  Teslow,  &  R.  Osman-Jouchoux,  The 
impact  of  constructivism  on  ID  fundamentals,  in  B.  Seels  (Ed.),  Instructional  Design  Fundamentals:  A 
Review  and  Reconsideration.  Englewood  Cliffs  NJ:  Educational  Technology  Publications. 


Situated  instructional  design  could  mean  a  couple  of  different  things: 

1 .  The  kind  of  instructional  design  (ID)  that  rests  upon  a  situated-cognition  view  of  human  learning  and 
performance.  Situated  cognition  is  a  powerful  metaphor  for  human  learning  that  incorporates 
elements  of  everyday  cognition,  informal  learning,  authentic  learning  experiences,  and  cultural 
influences  (Brown,  Collins,  &  Duguid,  1989).  Many  ID  theorists  are  heavily  influenced  by  situated 
cognition  and  constructivism  (e.g.,  Wilson  &  Cole,  1991,  in  press),  resulting  in  corresponding 
changes  in  ID  prescriptions. 

2.  The  kind  of  ID  that  adapts  itself  to  the  constraints  of  particular  situations  in  ways  that  traditional  ID 
models  do  not.  Traditional  ID  is  sometimes  criticized  for  being  overly  proceduralized  and  rigid.  A 
situated  approach  to  ID  would  be  more  sensitive  to  local  conditions  in  prescribing  both  methods  and 
outcomes  for  instructional  design. 

I  have  in  mind  a  view  of  situated  ID  that  incorporates  both  of  these  notions.  Much  of  the  literature  on 
constructivist  ID  relates  to  the  first  emphasis,  while  methodological  critiques  (e.g.,  Winn,  1990;  Wilson, 
Jonassen,  &  Cole,  1993)  relate  more  to  the  second  emphasis.  Thus  situated  ID  may  be  seen  as  an  approach 
that  incorporates  a  constructivist,  situated  view  of  learning  and  expertise,  while  at  the  same  time  viewing 
the  ID  process  itself  in  situated  terms. 

My  purpose  in  this  paper  is  twofold: 

1 .  to  offer  several  reflections  about  the  relationship  between  design  and  implementation  of  learning 
environments  and  instructional  products;  and 

2.  to  offer  a  number  of  specific  recommendations  for  practicing  ID  from  a  situated/constnictivist 
perspective. 

SITUATED  DESIGN:  BLURRING  THE  DISTINCTIONS 

Several  points  of  reflection  are  offered  in  this  section  pertaining  to  the  practice  of  instruction  design  as  it 
relates  to  real-world  contexts. 

1  .Implementation  and  design  are  ultimately  inseparable.  Some  of  the  most  valuable  lessons 
learned  about  instructional  design  comes  from  the  experience  gained  in  setting  up  and  administering 
programs.  Real-world  implementation  can  be  just  as  important  as  theory-guided  design. 
An  example  may  be  taken  from  the  1970s  research  in  computer-assisted  instruction.  One  program,  the 
TICCIT  project,  was  shown  by  an  NSF  evaluation  to  achieve  its  objectives  more  successfully  than 
traditional  classroom  im.truction  (Merrill,  Schneider,  &  Fletcher,  1979).  The  program  failed,  however,  in 
getting  students  to  stay  with  the  program;  the  dropout  rate  was  unacccptably  high  when  compared  to 
traditional  classrooms.  In  this  case,  the  actual  development  and  tryout  of  a  program  produced  knowledge 
that  was  not  anticipated  ahead  of  time. 

Another  example  is  Clancey's  Guidon-Manage  research  in  intelligent  tutoring  systems  (ITS).  In  a 
remarkable  example  of  self-reflection.  Clancey  (1993)  concludes:  "After  more  than  a  decade,  1  felt  that  1 
could  no  longer  continue  saying  that  I  was  developing  instructional  programs  for  medicine  because  not  a 
single  program  1  worked  on  was  in  routine  use..."  (p.  7).  This  lack  of  successful  implementation  was 
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attributable  to  the  researchers'  emphasis  on  the  design  of  systems  at  the  expense  of  meeting  the  needs  of 
identified  groups  of  practitioners.  While  Clancey  and  colleagues  were  successful  in  the  development  of 
improved  methods  of  knowledge  representation,  they  clearly  failed  in  getting  practitioners  to  notice. 
"[RJesearchers  must  participate  in  the  community  they  wish  to  influence....[S)omc  members  of  our  research 
community  must  necessarily  broaden  their  goals  from  developing  representational  tools  to  ch.  nging 
practice — changing  how  people  interact  and  changing  their  lives...  (p.  9,  italics  retained).  Clancey  reflects 
on  how  he  might  approach  the  Guidon-Manage  research  differently  today: 

•  participating  with  users  in  multidisciplinary  design  teams  versus  viewing  teachers  and  students  as  my 
subjects, 

•  adopting  a  global  view  of  the  context. ..instead  of  delivering  a  program  in  a.. .box, 

•  being  committed  to  provide  cost-effective  solutions  for  real  problems  versus  imposing  my  research 
agenda  on  another  community, 

•  facilitating  conversations  between  people  versus  only  automating  human  roles,... 

•  relating... ITS  computer  systems  to. ..everyday  practice.. .versus  viewing  models. ..as  constituting  the 
essence  of  expert  knowledge  that  is  to  be  transferred  to  a  student,  and 

•  viewing  the  group  as  a  psychological  unit  versus  modeling  only  individual  behavior.  (Clancey, 
1993,  p.  17) 

Clancey's  ITS  work  was  research  with  a  design  focus.  Although  the  specific  agenda  is  different,  the  lessons 
Clancey  learned  apply  very  well  to  users  of  ID  models.  Instructional  designers,  too,  should  "participate  in 
the  community  they  wish  to  influence." 

Like  the  TICCIT  and  Guidon  projects,  design  innovations  sometimes  result  in  negative  outcomes.  Failures 
of  ID  projects,  however,  become  particularly  valuable  items  because  they  can  indicate  areas  of  weakness  not 
just  in  the  current  project  but  in  the  ID  model  underlying  the  project.  Information  gleaned  from  negative 
findings  can  serve  as  formative  evaluation  data,  feeding  back  into  future  implementations  of  the  model  and 
into  future  version  of  the  model  itself. 

l.Questions  of  curriculum  and  value  are  central.  Perkins  (1992)  warns  of  a  common  fallacy 
implied  by  the  statement:  "What  we  need  is  a  new  and  better  method.  If  only  we  had  improved  ways  of 
inculcating  knowledge  or  inducing  youngsters  to  learn,  we  would  attain  the  precise.. .outcomes  we  cherish" 
(p.  44).  Instead,  Perkins  believes  that  "given  reasonably  sound  methods,  the  most  powerful  choice  we  can 
make  concerns  not  method  but  curriculum — not  how  we  teach  but  what  we  choose  to  try  to  teach"  (p.  44). 
This  comment  reaffirms  that  a  fundamental  step  in  instructional  design  involves  the  serious  consideration  of 
learning  goals.  A  variety  of  constituencies  should  be  included  in  this  process,  including  sponsors  and 
members  of  the  learning  community  itself.  Once  consensus  is  reached  about  the  kind  of  learning  being 
sought,  certain  teaching  models  become  unfeasible  while  others  become  more  attractive. 

As  Reigeluth  (1983)  acknowledges,  curriculum  and  instruction  cannot  be  completely  separated.  There  is  a 
tendency  among  many  institutions  to  give  lip  service  to  higher-order  outcomes  while  maintaining  teaching 
methods  that  specifically  suppress  such  outcomes.  Medical  schoois  that  teach  students  to  simply  memorize 
and  take  tests  are  an  example.  Another  example  is  a  military  school  whose  mission  statement  prizes 
"creativity"  in  students,  yet  whose  teaching  mehtods  and  authoritarian  culture  strictly  reinforce  conformity 
and  transmission  of  content. 

A  basic  lesson  learne  from  observing  schools  is  that  two  teachers  may  be  covering  the  same  ostensive 
curriculum  while  what  really  is  taught  differs  raically  between  them.  At  its  base,  the  construed vist 
movement  in  education  involves  curriculnm  reform,  a  rethinking  of  what  it  means  to  know  something.  A 
constructivist  curriculum  is  reflected  in  many  of  the  design  innovations  being  discussed  in  the  literature 
(Hanaffin,  1992).  Thus,  if  a  commitment  is  made  toward  rethinking  curriculum  to  expand  the  roles  of 
knowledge  construction  and  learning  communities,  then  a  corresponding  commitment  needs  to  be  made  in 
rethinking  learning  activities.  Deciding  upon  a  teaching  strategy  is  not  a  value-neutral  activity. 
Recognizing  this  puts  the  selection  of  an  ID  model  squarely  into  the  political  realm  of  policymaking.  New 
issues  become  important,  such  as  access,  equ'W.  representation,  voice,  and  achieving  consensus  amid 
diverse  perspectives. 


3.  Deciding  upon  a  design  solution  and  making  decisions  within  that  framework  is  a  highly  situated 
activity.  The  success  of  a  given  implementation  will  depend  moi'e  on  the  local  variables  than  on  the 
general  variables  contained  in  the  ID  model  chosen  to  guide  design.  Put  another  way,  "the  devil  is  in  the 
details."  There  is  a  way  to  succeed  and  a  way  to  fail  using  a  whole  host  of  teaching  models.  Teachers  and 
students  must  see  the  sense  of  what  they  are  doing,  come  to  believe  in  the  efficacy  of  the  program,  and 
work  hard  to  ensure  that  the  right  outcomes — that  is,  outcomes  that  are  consensually  agreed  upon — are 
achieved. 

This  situational  perspective  conflicts  with  traditional  views.  Thinking  of  instructional  design  as  a 
technology  would  lead  us  to  think  that  a  situation  gets  analyzed,  which  leads  to  a  technical  fix  to  be 
implemented,  which  leads  either  to  a  measured  solution  to  the  problem  or  a  revision  in  the  fix  for  the  next 
cycle  of  inten  irntion.  A  situated  view  of  instructional  design  would  lead  to  a  different  process,  something 
perhaps  like  this: 

1 .  A  learning  community  examines  and  negotiates  its  own  values,  desired  outcomes,  and  acceptable 
conventions  and  practices. 

2.  The  learning  community  plans  for  and  engages  in  knowledge -generating  activities  within  the 
established  framework  of  goals,  conventions,  and  practices. 

3.  Members  of  the  learning  community,  including  both  teachers  and  students,  observe  and  monitor 
learning  and  make  needed  adjustments  to  support  each  other  in  their  learning  activities. 

4.  Participants  occasionally  re-examine  negotiated  learning  goals  and  activities  for  the  purpose  of 
improving  learning  and  maintaining  a  vital  community  of  motivated  learners.  This  may  lead  to  new 
goals  and  methods  and  cultural  changes  at  all  levels,  from  cosmetic  to  foundational. 

This  situated,  community-oriented  view  of  instruction  takes  a  more  holistic  view  to  the  design  of 
instruction.  The  community  is  opportunistic  in  addressing  "design"  issues  at  any  stage  of  planning  and 
implementation.  Community  members,  including  students,  have  some  voice  in  determining  what  happens 
to  them  in  instruction.  In  return,  they  must  show  the  needed  commitment  and  disposition  to  behave 
responsibly  and  in  support  of  learning. 

If  community  members  have  participated  in  the  establishment  of  a  program,  they  are  more  likely  to  believe 
in  it.  If  they  believe  in  the  program,  the  chances  of  success  increase  dramatically.  As  Perkins  (1992) 
suggests,  even  very  imperfect  instructional  methods  can  work  if  the  commitment  is  made  to  work  together 
and  ask  the  right  questions  in  designing  curriculum. 

analysis — implicit  or  explicit — happens  in  designing  educational  programs.  A  survey  of 
instructional  models  reveals  that  some  have  demonstrably  high  development  costs.  Sherlock  ,  an 
electronics  troubleshooting  tutor  developed  for  the  military,  has  taken  a  decade  of  patient  research  to  achieve 
its  present  form  (Gott,  Lesgold,  &  Kane,  in  press),  resulting  in  impressive  learning  gains.  Once  developed, 
however,  the  program  may  be  replicated  at  a  reasonable  cost.  Other  models,  such  as  problem-based 
learning,  may  pose  heavy  demands  in  terms  of  time  in  the  curriculum  (cf.  Savery  &  Duffy,  in  press). 
Instructional  designers  (or  learning-community  members)  must  then  face  the  question  of  how  and  whether 
to  implement  such  resource-demanding  teaching  methods  into  an  existing  system  and  curriculum. 

Every  decision  to  adopt  one  teaching  approach  over  another  involves  such  weighing  of  pros  and  cons. 
However,  while  costs  may  be  objectively  measured  and  estimated,  learning  benefits  are  notoriously  difficult 
to  reduce  down  to  a  number.  This  inequity  of  measurability  results  in  a  common  bias:  The  cost  differences 
become  exaggerated  while  the  potential  benefits,  because  they  are  harder  to  measure,  tend  to  be  undervalued 
or  ignored.  Comparison  of  alternative  teaching  models  must  give  full  consideration  to  qualitative 
differences  in  learning  outcomes  in  addition  to  the  more  visible  cost  differences  in  time  and  money. 

Some  ideas  may  be  borrowed  and  inexpensively  incorporated  into  relatea  products  or  programs.  For 
example,  if  an  instructor  becomes  excited  by  Schank's  case-based  scenarios  (Schank,  Fano,  Bell,  &  Jona, 
1993/94),  she  may  choose  to  incorporate  case  histories  and  classroom  simulations  into  her  teaching.  While 
the  resulting  lessons  may  bear  only  a  passing  resemblance  to  the  computer-based  scenarios,  they  are  heavily 
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influenced  by  Schank's  principles  of  case-based,  interactive  instruction.  Many  important  design  principles, 
including  those  of  cognitive  apprenticeships  and  intentional  learning  communities,  can  be  efficiently 
adapted  into  instruction  in  a  number  of  ways,  depending  on  local  circumstances  and  resources. 

5. Instruction  should  support  learners  as  they  become  efficient  in  procedural  performance  and 
deliberate  in  their  self-reflection  and  understanding.  Most  current  ID  theories  emphasize  the  grounding  of 
instruction  in  complex  problems,  cases,  or  performance  opportunities.  Yet  organizing  instruction  around 
problems  and  cases  should  not  mask  the  importance  of  perception,  reflection,  and  metacognitive  activity. 
Indeed,  these  two  aspects  of  human  performance  (problem  solving  and  perception)  can  be  seen  as  inherently 
complementary  and  equally  necessary.  Experts  are  more  than  mere  automatic  problem-solvers.  Rather, 
experts  become  experts  through  a  progressive  series  of  encounters  with  the  domain,  each  involving  an 
element  of  routine  performance  and  a  corresponding  element  of  reflection  and  deliberation.  This  is  the 
process  of  expertise  spoken  of  by  Scardamalia  and  Bereiter  (1994). 

Pravvat  (1993)  makes  this  point  well.  While  there  is  a  tendency  among  cognitive  psychologists  to  make 
problem  solving  central  to  all  cognition,  Prawat  reminds  us  that  schemas,  ideas,  and  perceptual  processes 
hold  an  equally  important  place.  Learning  how  to  see  — from  a  variety  of  points  of  view — is  as  important 
as  solving  a  problem  once  we  do  see.  Principles  of  perception,  whether  from  ecological  psychology  (Allen 
and  Ott,  in  press),  connectionism,  or  aesthetics,  need  to  have  a  place  within  successful  ID  models.  This 
includes  teaching  students  how  to  represent  problems  and  situations,  but  also  how  to  appreciate  and  respond 
to  the  aesthetic  side  of  the  subject,  how  to  reflect  upon  one's  actions,  and  how  to  "raise  one's 
consciousness"  and  recognize  recurring  themes  and  patterns  in  behavior  and  interactions. 

^Successful  programs  must  seek  to  make  complex  performance  do-able  while  avoiding  the  pitfalls 
of  simplistic  proceduralization.  The  art  of  "scaffolding"  complex  performance  is  a  key  problem  area  that 
surprisingly  is  still  not  well  understood.  How  does  a  coach  entice  a  young  gymnast  to  perform  just  beyond 
her  capacities,  overcoming  the  fear  and  uncertainty  that  normally  accompany  new  performances?  How  does 
the  coach  know  just  when  and  where  to  step  in,  preserving  the  integrity  of  the  task  (and  the  learning)  while 
not  letting  the  athlete  fall  on  her  head?  These  are  questions  of  appropriate  scaffolding  or  support  for 
learning.  Once  a  teacher  begins  believing  the  constructivist  agenda  and  the  importance  of  authentic, 
meaningful  tasks,  then  the  challenge  of  supporting  novice  performance  within  a  complex  environment 
becomes  a  central  concern.  As  Sweller's  (1989)  research  makes  clear,  poorly  supported  problem-solving 
activities  force  learners  to  rely  on  weak  methods  that  they  already  know.  The  result  is  a  lot  of  wasted  time 
and  frustrated  learners.  Appropriate  and  wise  scaffolding  makes  problem-solving  activities  more  efficient 
because  learners  stay  focused  within  the  critical  "development"  zone  between  previously  mastered  knowledge 
and  skills  beyond  their  reach  (Vygotsky,  1978).  Developing  a  technology  for  optimizing  this  kind  of 
support  is  an  area  in  need  of  further  research  and  development. 

This  same  concept  of  scaffolding  can  be  directed  to  the  implementation  of  ID  models  themselves. 
Instructional  designers  and  teachers  need  proper  supports  and  aids  in  designing  according  to  a  particular 
model  or  tradition.  At  the  same  time,  they  should  be  cautioned  against  simplistically  "applying"  a  model 
in  a  proceduralized  or  objectivist  fashion.  Postmodernists  would  say  that  in  such  cases,  the  model  "does 
violence"  to  the  situation.  The  complexities  of  a  situation  should  not  be  reduced  down  to  the  simple 
maxims  of  a  teaching  model.  Any  model  that  is  forced  upon  a  situation  and  made  to  fit,  will  lead 
inevitably  to  unintended  negative  consequences.  The  negative  fallout  will  happen  at  those  points  of 
disjuncture  or  lack  of  fit  between  model  and  situation.  As  I  have  stressed,  the  details  of  the  situation  need 
to  be  respected  and  taken  into  account  when  adapting  a  model  to  a  situation. 

This,  perhaps,  is  a  more  appropriate  way  of  thinking  about  implementation:  Rather  than  applying  a 
particular  instructional  theory,  a  teacher  necessarily  adapts  that  theory  to  present  circumstances.  Learning 
how  to  adapt  abstractions  to  concrete  realities  is  a  worthy  task  for  both  students  and  teachers,  and  indeed, 
may  lie  at  the  heart  of  some  forms  of  expertise   This  process  of  adaptation  and  using  conceptual  models  as 
tools  in  a  given  situation  is  an  essential  ingredient  of  a  situated  approach  to  instructional  design. 
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MANAGING  CONSTRAINTS  TO  DESIGN 

Consider  what  it  means  to  design  something  (e.g.,  to  fashion  something  from  a  well-developed  plan).  ID 
Shares  with  a  design  activities  the  challenge  of  creating  something  that  accomplishes ;  a  g. ven  purpose 
within  me  constraints  and  parameters  of  the  situation.  Constraints  are  a  natural  part  of  the  creauve  des  gn 
Scess  desp  te  our  yearnings  for  unlimited  budgets,  motivated  learners,  and  relaxed  deadlines.  The  realit.es 
o  the  itua  ion  the  goals  of  instruction,  and  limited  resources  constitute  the  "raw  material"  from  which 
effective  dSs Zi 'take  shape  (Wedman  &  Tessmer,  1991).  Failure  to  consider  key  constraints  can  result 
in  the  failure  of  a  project. 

On  the  other  hand,  ID  sometimes  imposes  unnecessary  constraints  upon  itself  (Thiagarajan,  1976; 
Rowland  1991)    s  ID  always  served  by  a  plodding,  linear  methodology,  a  ngid  taxonomy  ot  learning 
IZoZs  or  a  fixed  pool  of  instructional  strategies?  Such  internally  imposed  constants  can  become  an 
oo stacle  to  creati  v  ty  and  an  unnecessary  burden  to  the  practitioner  and  to  learners.  Rap.d  prototyping 
HT  inn  &  Bichelmeyer  1990)  is  an  example  of  an  innovation  that  changes  the  sequence  of  design  steps 
aUowint  th Sner  to  redefine  ID  processes  to  better  suit  the  situation  and  the  tools  available.  The  tnck, 
of  cS,  is  knowTng  which  constraints  are  genuine  and  which  can  safely  be  discarded  as  new  possibilities 
present  themselves. 

TmHitinnil  ID  models  succeed  largelv  because  they  help  in  the  management  of  a  team  of  workers  engaged  in 
Icompl  x  projelf  The  cS  management  functions  of  monitoring  work  and  ensuring  account,  .ty  are 
nanTd  b/aS  of  checkpoints  or  signoffs-with  little  regard  for  their  impact  on  the  tojgB.^ 
Indeed  manaeement  goals  and  design  goals  are  often  in  tension  with  each  other.  For  an  ID  model  to  work 
^™S^5STmu«t  combine  these  two  critical  functions  into  a  workable  methodology:  effective, 
creative  design  and  efficient  management  and  control,  as  illustrated  below: 


FfWtivp  Creative  Design 

What  do  the.  learners  learn— really? 
Are  learners  motivated  by  instruction?  Do  they 
see  the  value  and  relevance  of  instruction? 
Do  learners  use  their  knowledge  to  solve 
problems  in  authentic  performance  settings? 
Arc  learning  environments  rich  in  information, 
guidance,  and  support? 


Efficient  Management  and  Control 

Will  the  instructional  product  reflect  a  return  on 
investment? 

Is  the  development  process  efficient?  Are 
resources  being  well  utilized  throughout  the 
design  process? 

Is  there  systematic  planning,  decision-making, 
and  accountability  in  the  design  process? 


~7U>le  1.  Two  competing  functions  of  instructional  design  models. 


but  remain  forever  in  a  stale  of  partial  completion. 
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Figure  1 .  Hot  air  balloon  analogy. 


Figure  l  illustrates  this  tension  by  reference  to  a  hot  air  balloon  trying  to  reach  upward,  but  being  tethered 
down  by  a  number  of  constraints — some  real,  some  artificial.  Ignore  the  constraints  entirely  and  project 
costs  rise,  into  the  stratosphere.  The  point  is  that  we  need  a  balanced  set  of  safeguards  and  constraints  that 
assure  careful  design  and  accountability  but  which  are  flexible  enough  to  allow  the  project  to  safely  "fly." 
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WHO  DOES  THE  DESIGN? 


A  key  element  in  effective  ID  is  the  nature  of  the  design  team.  Instead  of  a  designer  and  subject  expert 
working  in  relative  isolation,  situated  ID  suggests  that  all  major  constituencies  be  represented  on  the  design 
team,  including  teachers  and  students.  These  end  users — the  "consumers"  of  the  instructional  "product" — 
should  contribute  directly  to  the  project's  design  and  development.  Greenbaum  &  Kyng  (1991)  refer  to  this 
as  participatory  design,  and  Clancey  (1993)  recommends  "we  must  involve  students,  teachers, 
administrators,  future  employers,  and  the  community  as  participants  in  design...,  working  with  students  and 
teachers  in  their  setting — not  just  calling  them  into  the.. .lab  to  work  with  us"  (pp.  9,  20). 
An  instructional  designer  might  complain:  "But  we've  always  incorporated  the  end  user  in  our  ID  models; 
this  sounds  like  warmed-over  formative  evaluation."  I  would  respond:  "If  formative  evaluation  got  done  a 
tenth  as  much  as  it  gets  talked  about,  ID  practice  would  be  in  much  better  shape."  In  many  ways,  a 
constructivist  or  situated  approach  to  ID  takes  old  ideas  and  gives  a  new  impetus  to  them.  Consider  the 
traditional  roles  of  team  members: 

•  Subject  niatter  expert  (SME).  Provides  the  content  and  expertise. 

•  Designer.  Figures  out  a  way  to  extract  (milk?)  expertise  from  the  SME  and  encode  it  into 
instructional  materials.  Selects  instructional  strategy  appropriate  to  the  content  and  the  situation  to 
effectively  teach  the  content  to  the  learner. 

•  Teacher  and  student.  At  formative  evaluation  stages,  serve  as  subjects  for  tryout  tests  to  maximize 
usability  and  learnability.  At  implementation  stage,  teachers  and  students  take  the  instructional 
materials  and  carefully  use  them  as  directed.  Something  like  a  doctor's  prescription. 

Situated  ID  mixes  up  the  roles  much  more.  SMEs  can  help  design  learning  experiences;  designers  manage 
projects,  build  teams,  check  for  content  accuracy,  and  serve  as  model  learners  and  teachers.  Teachers  and 
students  may  help  define  or  select  content  and  design  their  own  learning  experiences.  Poorly  implemented, 
the  redefinition  and  blurring  of  roles  can  lead  to  chaos  and  confusion;  well  implemented,  a  flexible  team 
orientation  can  result  in  a  synergy  or  a  fusion  of  multiple  perspectives  that  improves  the  design. 


GUIDELINES  FOR  DOING  SITUATED/CONSTRUCTIVIST  ID 

This  section  is  composed  of  a  laundry  list  of  tips  for  viewing  ID  from  a  situated  perspective,  organized 
according  to  generic  ID  phases.  Issues  of  implementation,  discussed  at  the  beginning  of  the  paper,  are  not 
addressed  in  detail.  Some  of  the  tips  are  abstract  and  conceptual;  others  are  simple  and  practical.  Some 
depart  radically  from  current  practice;  others  reflect  how  most  practitioners  already  view  their  jobs. 
Collectively,  they  provide  a  clearer  picture  of  what  it  means  to  do  situated  ID. 


GENERAL  METHODOLOGY 

•  Include  end  users  (both  teachers  and  students)  as  part  of  the  design  team.  Incorporate  participatory 
design  techniques,  with  design  activity  moving  out  of  the  "lab"  and  into  the  field. 

•  Configure  a  design  model  appropriate  to  the  task  at  hand.  Several  constructivist  models  of 
instruction  present  six  or  seven  key  principles.  Why  not  let  a  design  team  themselves  select  or 
develop  the  principles  that  best  reflects  their  values?  Each  design  team  needs  to  configure  a 
model — cutting  and  pasting  from  extant  literature  and  practice — that  is  tailored  to  their  community 
and  the  constraints  of  the  project. 

NEEDS  ASSESSMENT 

•  Consider  solutions  that  are  closer  to  the  performance  context  (job  aids,  just-in-time  training, 
performance  support  systems,  etc.).  This  is  consistent  with  situated  models  of  cognition  and  with 
the  notion  of  distributed  cognition  (Perkins,  1993). 

•  Resist  the  temptation  to  be  driven  by  easily  measured  and  manipulated  content.  Many  important 
learning  outcomes  cannot  be  easily  measured. 
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•  Ask:  Who  makes  the  rules  about  what  constitutes  a  need?  Are  there  other  perspectives  to  consider? 
What  (and  whose)  need:  are  being  neglected?  These  questions  arise  out  of  the  postmodern  notion  of 
the  ideological  base  of  all  human  activity. 

GOAL/TASK  ANALYSES 

•  Use  objectives  as  heuristics  to  guide  design.  Don't  always  insist  on  operational  performance 
descriptions  which  may  constrain  (he  learners'  goals  and  achievement.  Pushing       ..tatements  to 
behavioral  specifications  can  often  be  wasted  work.  The  "intent"  of  instruction  can  be  made  clear  by 
examining  goal  statements,  learning  activities,  and  assessment  methods.  Goals  and  objectives 
should  be  specific  enough  to  serve  as  inputs  to  the  design  of  assessments  and  instructional 
strategies. 

•  Don't  expect  to  "capture"  the  content  in  your  goal-  or  task  analysii  .  Content  on  papei  not  the 
expertise  in  a  practitioner's  head,  hands,  and  feet.  The  best  analysis  always  falls  short  of  the  mark. 
The  only  remedy  is  to  design  rich  learning  experiences  where  learners  can  pick  up  on  their  own  the 
content  missing  between  the  gaps  of  analysis. 

•  Allow  for  instruction  and  learning  goals  to  emerge  during  instruction.  Just  as  content  cannot  be 
fully  captured,  learning  goals  cannot  be  fully  pre-specified  apart  from  the  actual  learning  context. 
See  Winn  (1990)  for  a  thorough  discussion  of  this  issue. 

Consider  multiple  stages  of  expertise.  Expertise  is  usually  thought  of  as  having  two  levels:  Expert 
or  proficient  performance  and  novice  or  initial  performance.  Of  course,  a  two-level  model  is 
insufficient  for  accurate  modeling  of  student  growth  over  time.  A  series  of  qualitative  models  of 
expertise  may  be  needed  for  modeling  students'  progression  in  learning  critical  tasks  (White  & 
Frederiksen,  1986).  Be  prepared  to  confront  learners'  naive,  intuitive  theories  and  to  scaffold  their 
learning. 

•  Give  priority  to  problem-solving,  meaning-constructing  learning  goals.  Instead  of  rule-following, 
emphasize  problem  solving  (which  incorporates  rule-following  but  is  not  limited  to  it).  Instead  of 
simple  recall  tasks,  ask  learners  to  make  sense  out  of  material  and  demonstrate  their  understanding 
of  it. 

•  Look  for  authentic,  information- rich  methods  for  representing  content  and  assessing  performance 
(e.g.,  audio,  video).  High-resolution  methods  for  representing  content  can  be  useful  throughout  the 
ID  process.  Whereas  we  usually  associate  audio  and  video  representations  only  with  presentation  of 
material  to  students,  the  same  representation  tools  may  be  useful  for  documenting  expertise  and 
assessing  student  understanding. 

•  Define  content  in  multiple  ways.  Use  cases,  stories,  and  patterns  in  addition  to  rules,  principles, 
and  procedures.  Rich  cases,  stories,  and  patterns  of  performance  can  be  alternative  metaphors  for 
finding  and  representing  content. 

Appreciate  the  value -ladenness  of  all  analysis.  Defining  content  is  a  political,  ideological 
enterprise.  Valuing  one  perspective  means  that  other  perspectives  will  be  given  less  value.  One 
approach  is  given  prominence;  another  is  neglected.  Somebody  wins,  and  somebody  loses.  Be 
sensitive  to  the  value  implications  of  your  decisions. 

Ask:  Who  makes  the  rules  about  what  constitutes  a  legitimate  learning  goal?  What  learning  goals 
are  not  being  analyzed?  What  is  the  hidden  agenda?  Twenty  years  ago,  a  designer  using 
"understanding"  in  a  learning  objective  would  have  been  laughed  out  of  the  office.  "Understanding" 
was  fuzzy;  it  was  forbidden.  Are  there  other  expressions  of  learning  outcomes  that  remain  taboo? 
Are  there  other  dimensions  of  human  performance  that  remain  undervalued?  Good  postmodern  ID 
would  pursue  answers  to  these  questions  and  be  unafraid  of  reexamining  current  practice. 

INSTRUCTIONAL  STRATEGY  DEVELOPMENT 

Distinguish  between  instructional  goals  and  learners'  goals;  support  learners  in  pursuing  their  own 
goals.  Ng  and  Bereiter  (1991)  distinguish  between  (1)  task-completion  goals  or  "hoop  jumping," 
(2)  instructional  goals  set  by  the  system,  and  (3)  personal  knowledge-building  goals  set  by  the 
student.  The  three  do  not  always  converge.  A  student  motivated  by  task-completion  goals  doesn't 
even  consider  learning,  yet  many  students'  behavior  in  schools  is  driven  by  performance 
requirements.  Constructivist  instruction  would  nourish  and  encourage  pursuit  of  personal 
knowledge-building  goals,  while  still  supporting  instructional  goals.  As  Mark  Twain  put  it:  "I 
have  never  let  my  schooling  interfere  with  my  education." 
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•  Allow  for  multiple  goals  for  different  learners.  ID  often  includes  the  implicit  assumption  that 
instructional  goals  will  be  identical  for  all  learners.  This  is  sometimes  necessary,  but  not  always. 
Hypermedia  learning  environments  almost  by  definition  are  designed  to  accommodate  multiple 
learning  goals.  Even  within  traditional  classrooms,  technologies  exist  today  for  managing  multiple 
learning  goals  (Collins,  1991). 

•  Appreciate  the  interdcpendency  of  content  and  method.  Traditional  design  theory  treats  content  and 
the  method  for  teaching  that  content  as  orthogonally  independent  factors.  Situated  ID  says  you  can't 
entirely  separate  the  two.  When  you  use  a  Socratic  method,  you  are  teaching  something  quite 
different  than  when  you  use  worksheets  and  a  posttest.  Teaching  concepts  via  a  rule  definition 
results  in  something  different  than  teaching  the  concept  via  rich  cases.  Just  as  McLuhan  discerned 
the  confounding  of  "media"  and  "message,"  so  designers  must  sec  how  learning  goals  are  not 
uniformly  met  by  interchangeable  instructional  strategies  (see  Wilson,  in  press  a). 

Look  for  opportunities  to  give  guided  control  to  the  learner,  encouraging  development  of 
metacognitive  knowledge.  Encourage  growth  in  students'  metacognitive  knowledge,  what  we  often 
call  "learning  how  to  learn."  Don't  assume  that  students  know  how  to  exercise  effective  learning 
control;  instead,  establish  metacognitive  skills  as  a  learning  goal  for  instruction  to  achieve. 
Allow  for  the  "teaching  moment. "'  Situations  occur  within  instruction  where  the  student  is  primed 
and  ready  to  learn  a  significant  new  insight.  Good  teachers  create  conditions  where  such  moments 
occur  regularly .  then  they  seize  the  moment  and  teach  the  lesson.  This  kind  of  flexibility  requires  a 
level  of  spontaneity  and  responsiveness  not  usually  talked  about  in  ID  circles. 
Think  in  terms  of  designing  learning  environments  rather  than  "selecting  "  instructional  strategies. 
Metaphors  are  important.  Does  the  designer  "select"  a  strategy  or  "culti » _ie"  a  climate  conducive  to 
learning?  Wilson  (in  press  b)  argues  for  learning  environments  to  complement  our  product, 
process,  and  systems  metaphors  for  instruction.  More  often  than  not,  teachers  and  designers  create 
or  adapt  an  instructional  strategy  rather  than  "selecting"  one. 

•  Think  of  instruction  as  providing  tools  that  teachers  and  students  can  use  for  learning;  make  these 
tools  user-friendly.  This  frame  of  mind  is  virtually  the  opposite  of  "teacher-proofing"  instructional 
materials  to  assure  uniform  adherence  to  designers'  use  expectations.  Instead,  teachers  and  students 
are  encouraged  to  make  creative  and  intelligent  use  of  instructional  tools  and  resources. 
Appreciate  the  value-ladenness  of  instructional  strategies.  Sitting  through  a  school  board  meeting  is 
enough  to  convince  anyone  of  this.  Instructional  strategies  grow  out  of  our  philosophies  of  the 
world  and  our  value  systems.  Not  only  the  content,  but  th»  strategy  can  be  a  threat  to  particular 
ideological  positions  or  to  learner  motivation.  Good  designers  will  be  sensitive  to  the  "fit"  between 
their  designs  and  the  situation. 

MEDIA  SELECTION 

Consider  media  factors  early  in  the  design  cycle.  Practical  and  cost  constraints  typically  dictate  thai 
tentative  media  decisions  will  be  made  relatively  early  in  the  design  process.  Media  then  becomes 
one  of  the  instructional  factors  that  receives  increasing  attention  through  iterations  of  analysis. 
Include  media  literacy  and  biases  as  a  consideration  in  media  decisions.  Different  media  send 
different  "messages"  to  an  audience,  independently  of  the  instructional  content.  Look  for  any 
"hidden  curriculum"  elements  in  different  media  choices.  Avoid  negative  stereotypes  and  cultural 
biases.  Consider  the  rhetorical  goodness  of  fit  between  media  choice  and  overall  instructional 
purposes.  Also,  design  messages  that  are  sensitive  to  an  audience's  media  sophistication  and 
literacy,  paying  particular  attention  to  humor,  media  conventions,  and  production  values. 

STUDENT  ASSESSMENT 

Incorporate  assessment  into  the  teaching  product  where  possible.  Technologies  are  available  for 
incorporating  continuous,  "dynamic  assessment"  into  learning  materials  (Lajoie  &  Lesgold,  1992). 
Assessment  can  then  be  seamlessly  integrated  into  meaningful  learning  experiences  and  not  tacked 
on  at  the  end  (Frederiksen  &  Collins  1989). 

Critique  and  discuss  products  grounded  in  authentic  contexts,  including  portfolios,  projects, 
compositions,  and  performances.  Use  of  work  products  can  complement  more  direct,  traditional 
measures  of  knowledge  acquisition  and  understanding  (Cates,  1992).  Include  different  perspectives 
in  the  critiquing  process. 
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Evaluate  processes  as  well  as  products.  The  cognitive  apprenticeship  model  offers  a  number  of 
strategies  for  reflecting  on  process:  debriefings,  abstracted  replays,  dramatizations,  interviews,  group 
discussions,  knowledge  telling,  co-investigation,  and  post-mortems  of  problem-solving  activities 
(Collins  &  Brown,  1987;  McLellan,  1993;  Gay  &  Mazur,  1993). 

Use  informal  assessments  within  classrooms  and  learning  environments.  Informal  assessments  refer 
primarily  to  teacher  observations  of  eye  contact,  body  language,  facial  expressions,  and  work 
performance.  These  observations  can  complement  formal  assessments  as  a  basis  for  instructional 
adjustments. 

At  this  point,  we  should  consider  the  pros  and  cons  associated  with  following  a  more  situated  model  of 
design.  Here  is  a  list  of  possible  advantages: 

more  meaningful  learning  outcomes  that  are  likely  to  be  used  in  relevant  contexts 
more  meaningful  participation  of  the  learner  in  the  learning  process 
more  independent  problem-solving  capability 
more  flexibility  in  design  activities 
more  flexibility  in  instructional  activities 

more  acknowledgment  of  social  and  motivational  factors  in  learning. 


Here  are  some  possible  risks: 

•  more  costly  instruction 

greater  need  for  instructional  resources  and  information  management 

•  less  coverage  of  material 

less  demonstration  of  specific  skill  mastery 

chaos  and  confusion  if  poorly  implemented. 

The  point  is:  (1)  we  really  don't  know  all  the  pros  and  cons  of  new  approaches,  oecause  we've  never  fully 
tried  them  out,  and  (2)  as  any  situationist  would  say,  it  depends  on  the  situation,  on  how  it's  done.  There 
are  good  ways  to  do  situated  ID  and  bad  ways,  just  as  one  can  point  to  excellent  and  poor  examples  of 
training  developed  with  an  objectivist  philosophy.  We  will  learn  more  about  the  real  pros  and  cons  ot 
doing  situated  ID  as  more  design  models  become  available  and  as  they  become  more  widely  used. 
Obviously,  the  details  of  situated  approaches  to  instructional  design  have  not  yet  been  thoroughly  worked 
out.  At  a  time  of  such  basic  re-thinking  about  the  nature  of  cognition,  it  is  hard  to  be  dogmatic  about  what 
teaching  strategies  comprise  the  "optimal"  design  in  any  subject  matter.  Perhaps  the  main  lesson  for  now 
is  that  the  discussion  should  be  followed  with  a  certain  degree  of  skepticism,  with  an  eye  toward  implica- 
tions for  professional  practice.  Our  knowledge  base  in  cognition  and  instructional  design  really  is  fragile, 
depending  on  a  shifting  foundation  that  will  likely  continue  to  change  in  the  years  to  come. 
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Comparing  the  effects  of  interactive  audio  in  Computer-Based  Instruction  (CBI)  versus 
non-interactive  audio  in  CBI  on  the  skills  and  attitudes  of  music  students  in  identifying 
harmonic  voicing  has  been  divided  into  two  sub-problems  for  the  purposes  of  this  study. 
The  first  sub-problem  is  to  compare  music  students'  skill  in  identifying  harmonic  voicing 
using  interactive  versus  non-interactive  audio.  The  second  sub-problem  is  to  compare 
music  students'  attitude  toward  CBI  using  interactive  versus  non-interactive  audio. 
A  two  group  pretest-posttest  experiment  was  conducted  in  which  forty-six  participants 
volunteered  to  work  with  a  computer  program  developed  for  this  study;  the  experimental 
group  used  interactive  audio  and  the  control  group  used  non-interactive  audio;  and,  both 
groups  completed  an  attitude  measure.  The  test  scores  showed  acceptable  to  high 
reliability.  Analysis  of  the  data  revealed  that  the  experimental  group  showed  greater  skill 
development  than  the  control  group  in  identifying  harmonic  voicing.  Analysis  of  the 
attitudinal  measurement  revealed  no  difference  between  the  groups. 


The  subject  area  of  study  is  aural  skills  training  in  music.  The  musician  in  ensemble  performance  is 
typically  surrounded  by  harmony.  A  good  performance  is  partially  dependent  upon  proper  relationships 
among  voices.  Effective  harmonic  analysis  provides  the  foundation  for  understanding  the  interrelationships 
between  voice  parts:  identification  of  individual  voices  which  make  up  the  vertical  harmonic  structure 
requires  abilities  in  listening  and  analysis.  Although  music  and  music  education  may  initially  appear  to  be 
quite  self-contained  without  the  need  for  technology,  modern  technology  can  be  very  valuable  to  both  the 
modern  musician  and  the  music  teacher.  Using  CBI  for  aural  training  in  music  implies  the  use  of  audio:  it 
is  not  the  purpose  of  this  study  to  examine  the  use  of  audio  in  CBI  but  to  examine  the  implications  of 
allowing  user-control  of  specific  parameters  of  the  audio.  Hofstetter  pioneered  the  research  supporting  CBI 
using  the  PLATO  system  with  a  program  called  GUIDO  (Hofstetter,  1975,  1979).  Taylor  (1982)  evaluated 
computerized  melodic  dictation  using  PLATO  and  a  program  called  MEDICI.  He  recommended  that 
MEDICI  be  used  at  the  university  level  of  study  and  suggested  further  developments  for  CBI  in  aural  skills. 
Interactive  audio  in  computer-based  aural  skills  instruction  has  not  been  thoroughly  documented:  GUIDO 
allowed  student  :ontrol  of  individual  voices  (Arenson  &  Hofstetter,  1983)  but  the  effectiveness  of  this 
interactivity  was  never  evaluated.  Woodrulff  &  Heeler  (1990,  1991)  and  Ashley  (1989)  successfully  used 
interactive  audio  with  videodisk  and  CD-ROM  technology  to  develop  listening  skills  of  music  students. 
Aural  skills  need  to  be  developed  in  order  for  musicians  to  experience  music  more  completely.  Given  that 
involving  the  student  in  interactive  dialog  during  analysis  can  help  form  better  listeners  and  belter 
musicians  (Pogonowski,  1989)  and  that  computer  technology  with  multimedia  has  been  successfully 
applied  in  music  education  (Ashley,  1989;  Gillespie  &  Placek,  1991 ;  Kuzmich,  1989;  Woodrulff  &  Heeler. 
1990,  1991),  CBI  in  aural  skills  with  multimedia  presentation  in  an  interactive  environment  may  lead  to 
increased  student  abilities  in  harmonic  analysis.  Interactivity  in  this  context  refers  to  audible  samples  of 
three  part  harmonies  utilizing  user  control  of  individual  voice  volume. 

Review  of  Related  Literature 

Identification  of  harmonic  voicing  is  closely  related  to  harmonic  dictation  and  is  one  of  many  fundamental 
aural  skills.  Aural  skills  remain  an  important  foundation  for  musicians.  The  term  refers  to  skill  in 
recognition  or  identification  of  various  components  of  music.  Some  of  these  components  are:  intervals, 
melodies,  chords,  harmonies,  rhythms,  timbres,  and  dynamics.  Listening  for  all  these  components  at  the 
same  time  is  very  complicated  (Madsen  &  Geringer,  1990)  and  therefore  either  very  sophisticated 
measurement  is  necessary,  or  individual  components  should  be  used  for  training  and  evaluation.  Studies  in 
aural  skills,  both  with  and  without  computer  aid,  have  looked  at  many  of  these  individual  components. 

Computer-Based  Instruction 

A  wealth  of  literature  exists  which  shows  the  merits  of  using  CBI  in  education.  CBI  can  offer 
individualized  instruction.  Bloom  (1984)  describes  methods  of  instruction  and  concludes  that  individualized 
instruction,  specifically  one-to-one  tutoring,  results  in  the  greatest  improvement  in  achievement.  Kulik, 
Kulik  and  Cohen  (1980)  conclusively  show  that  the  use  of  CBI  results  in  increased  achievement.  By  the 
mid  1970's  main-frame  computers  were  supporting  aural  skills  training  programs  (Hofstetter,  1975),  and 
research  was  indicating  positive  results  of  CBI  in  music.  The  capability  of  the  computer  (mainframe  and 
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micro)  to  generate  and  play  music  provides  a  means  for  aural  skills  training  on  an  individual  basis  without 
the  costs  and  difficulties  of  human  tutoring  or  individualized  classroom  instruction. 

Aural  Skills  and  CBl 

One  program  for  aural  skills  training  is  the  source  for  some  of  the  pioneering  research  in  this  area.  GUIDO 
is  a  system  which  was  developed  beginning  in  1974  for  operation  on  a  PLATO  main-frame  (Hofs'.ctter, 
1 975,  1981;  Arenson  &  Hofstetter,  1 983).  A  series  of  four  studies  all  supported  the  use  of  CBl  for  aural 
skills  training  (Hofstetter,  1978,  1979,  1980,  1981)  and  recommended  further  study  in  the  area. 
In  addition  to  the  studies  by  Hofstetter,  other  research  supports  the  use  of  CBl  in  aural  skills.  Canclos, 
Murphy,  Blombach  and  Heck  (1980)  evaluated  mastery  learning,  linear  instruction  and  seif-practice  as 
methods  for  teaching  interval  identification.  Their  results  show  that  mastery  learning  and  linear  instruction 
(both  CBl)  are  superior  to  the  traditional  method  in  terms  of  achievement.  Other  studies  compared  CBl 
competency  based  learning  of  aural  skills  to  traditional  methods  (Arenson,  1982;  Greenfield  &  Codding, 
1985)  and  found  significant  differences  in  achievement  in  favour  of  CBl.  Another  study  on  CBl  in  aural 
skills  was  done  using  MEDICI,  a  system  developed  in  1980  for  operation  on  a  PLATO  main-frame 
(Newcomb,  Weage  and  Spencer,  1981;  Taylor,  1982).  Taylor  found  no  difference  in  achievement  between 
CBl  and  traditional  instruction  in  melodic  dictation,  however  CBl  proved  to  be  more  efficient  and  was 
recommended  for  further  use.  A  more  recent  study  comparing  CBl  and  traditional  instruction  in  music 
education  was  done  by  Bailey  (1 989)  and  similar  results  were  found:  CBl  in  music  results  in  higher 
achievement. 

Interactive  Multimedia 

Woodrulff  and  Heeler  (1990)  compared  the  use  of  CBl  and  interactive  videodisk  versus  CBl  and  no 
vidcodisk.  In  the  study,  video  capabilities  were  not  used:  at  the  time  of  the  study  videodisk  technology  was 
simply  more  accessible  than  CD-ROM  technology,  an'  therefore  was  used  as  an  audio  source.  The  study 
showed  increased  achievement  as  a  result  of  interactive  ^BI.  Where  previous  studies  usually  compared  CBl 
with  traditional  instruction,  Woodrulff  and  Heeler  compared  two  groups  which  both  use  CBl,  differing  only 
in  terms  of  interactivity.  Ashley  (1989)  used  CD-ROM  technology  for  developing  listening  skills  in  music 
students.  The  study  reported  that  interactive  CD-ROM  was  effective  in  honing  listening  skills  with  respect 
to  rhythm,  harmony,  melody,  form  and  timbre.  These  studies  (Ashley,  1989;  Woodrulff  &  Heeler,  1990) 
support  further  research  into  interactive  applications  in  computer-based  music  instruction.  With  regard  to 
interactivity,  Taylor  (1988)  commented  that  "students  will  become  more  involved  when  they  can  interact 
with  their  learning  environment  in  interesting  and  different  ways,  and  they  will  also  have  the  opportunity  to 
be  creative  in  their  personal  approaches  to  learning"  (p.  54). 

Altitudes  Towards  CBl  and  Aural  Skills 

Many  studies,  in  addition  to  evaluation  of  achievement,  also  include  attitudinal  surveys.  The  majority  of 
studies  dealing  with  CBl  and  aural  skills  report  that  students  have  positive  attitudes  toward  CBl  in  music: 
students  show  increased  interest,  enthusiasm  and  motivation  in  music  studies  (Canelos,  Murphy,  Blombach 
&  Heck,  1980;  Clarkson  &  Pegley,  1991;  Gross  &  Griffin,  1982;  Hoffman,  1991;  Hofstetter,  1979; 
Upitis,  1982;  Weintraub,  1991 ).  One  study  reported  that  no  change  in  attitude  occurred  during  the 
experiment  (Humphries,  1980),  however,  the  study  does  not  indicate  if  the  attitudes  toward  CBl  in  aural 
skills  were  positive  or  negative. 

A  notable  exception  identifies  a  majority  of  students  who  indicate  that  they  felt  CBl  lessons  in  aural  skills 
were  too  time  consuming  (Greenfield  and  Codding,  1985).  These  findings  are  supported  by  Pembrook 
(1986)  who  found  that  the  majority  of  students  who  had  used  MEDICI  reported  a  negative  experience. 
Positive  comments  were  made  regarding  the  difficulty  level  of  content,  individualization,  schedule 
flexibility,  immediate  grading  and  positive  feedback,  however  the  majority  of  students  complained  about  the 
time  commitment  needed  for  CBl. 

The  majority  of  attitudinal  surveys  report  general  positive  attitude  toward  CBl  although  some  exceptions 
exist.  Generally,  student  attitude  seems  to  favour  CBl  in  music  aural  skids. 
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Summary 


Since  much  research  points  to  the  conclusion  that  CBI  is  effective  and  efficient,  subsequent  research  has  not 
bothered  with  the  comparison  of  CBI  and  traditional  instruction  but  instead  has  concentrated  on  variables 
within  the  CBI  environment.  CBI  in  aural  skills  has  been  effective  both  in  terms  of  achievement  and 
student  attitude.  Continuation  of  CBI  applications  in  music  is  warranted  and  less  developed  areas  such  as 
harmonic  voice  identification,  which  have  not  been  well  documented,  should  be  studied.  This 
recommendation  combined  with  the  support  for  interactive  CBI  prepare  sufficient  groundwork  for  this  study. 
Newcomb  (1988)  summarizes  and  supports  CBI  in  music: 

How  are  we  doing  in  computer-based  music  instruction?  We  have  made  a  beginning.  The  vineyard  is  large, 
the  workers  are  few,  and  budgets  are  small.  The  work  is  fascinating.  Public  interest  is  growing,  and  the 
long-term  outlook  is  partly  sunny,  (p.49) 

The  Current  Study 

Statement  of  the  Problem 

To  compare  the  effects  of  interactive  audio  in  CBI  versus  non-interactive  audio  in  CBI  on  the  skills  and 
attitudes  of  music  students  in  identifying  harmonic  voicing. 
Two  sub-problems  were  used: 

1)  to  compare  music  students'  skill  development  in  identifying  harmonic  voicing  using  interactive  audio  in 
TBI  with  those  using  non-interactive  audio  in  CBI;  and, 

2)  to  compare  music  students'  attitudes  toward  CBI  in  aural  skills  and  interactive  audio  with  those  toward 
CBI  in  aural  skills  and  non-interactive  audio. 

Method 

The  participants  were  a  volunteer  sample  from  an  accessible  population  of  university  music  students 
studying  aural  skills.  Approximately  ninety-five  students  were  eligible  for  the  study  and  forty-six 
volunteered  to  participate.  A  non-commercial  computer  program  developed  specifically  for  the  experiment 
was  used  to  deliver  the  lesson  and  gather  data.  All  participants  completed  the  pretest,  experimental 
treatment,  posttest  and  attitude  survey  in  one  session,  lasting  approximately  one  hour,  on  the  same 
equipment.  The  study  included  musical  content  consisting  of  differentiation  of  major,  minor  and  diminished 
triads. 

The  experiment  was  structured  as  a  pretest-posttest  control  group  design  :  similar  in  design  to  Hofstetter 
(1979)  and  Woodrulff  &  Heeler  (1990).  Each  participant  was  randomly  assigned  to  either  the  control  group 
or  the  treatment  group.  Although  both  groups  used  the  same  program,  the  experimental  group  (interactive 
audio)  had  control  over  individual  voice  volumes  in  the  audio  component;  whereas,  the  control  group  used 
computer-controlled  voice  volumes  in  the  audio  component.  All  participants  were  able  to  hear  the  audible 
samples  multiple  times  upon  request.  As  the  participants  progressed  through  the  sequence  of  the  program, 
relevant  data  were  recorded  in  an  external  file. 

Instrumentation 

A  computer  program  in  four  parts  developed  with  Authorware  Professional  ©  version  1 .7  for  Macintosh 
was  used  to  deliver  the  lesson  and  colLct  data.  This  platform  was  chosen  because  of  the  familiarity  the 
aural  skills  students  had  with  Macintosh  computers  from  use  in  the  aural  skills  course.  The  program  was 
developed  carefully  with  input  from  both  instructional  designers  and  content  experts.  Two  customized 
features  were  used  with  Authorware  Professional:  an  external  command  (XFCN)  was  built  to  play  the  triads 
(SndCmd  XFCN  uses  control  parameters  to  generate  wave  patterns  at  defined  frequencies  using  the  internal 
tone-generators  in  the  Macintosh,  using  System  7)  and  a  modified  version  of  the  RunAPM  file  was  used  to 
deliver  the  program  (the  "return"  icon  used  in  Run  APM  version  1.7A.1  was  modified  to  match  the  arrow 
icons  used  in  the  program). 
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Results  for  the  First  Sub-problem 

To  determine  skill  development  in  identifying  harmonic  voicing,  individual  scores  on  the  aural  harmony 
pretest  and  posttest  were  tabulated.  The  numbers  are  raw  scores  out  of  twenty,  each  set  corresponding  to 
one  individual  from  either  the  experimental  or  control  group.  Summaries  of  these  scores  can  be  found  in 
Table  !  and  Figure  1 . 

Table  I . 


Test  scores  of  identification  of  harmonic  voicing. 


ri 

Pretest 
Mean 

Posttest 
Mean 

Overall 
Mean 

Pretest 
SD 

Posttest 
SD 

Overall 
SD 

Experimenta 

22 

13  .  6 

14  .  9 

14  .3 

3  .  9 

3  .  8 

3  .  8 

Control 

24 

13  .  5 

14  .  5 

14  .  0 

3  .  6 

3  .  3 

3  .  5 

All  Subject 

3  46 

13  .  5 

14  .7 

14  . 1 

3  .7 

3  .  5 

3  .  6 

Figure  1. 

Test  scores  of  identification  of  harmonic  voicing 


Pretest  Posttest  Overall  Pretest  Posttest  Overall 
Mean     Mean     Mean       SD        SD  SD 


Reliability  coefficient  alpha  was  0.72  for  the  twenty  item  pretest  indicating  that  the  test  was  internally 
consistent  and  reliable.  The  posttest  was  structured  identical  to  the  pretest,  however  the  audible  samples 
were  different  and  the  correct  responses  therefore  were  also  different.  Coefficient  alpha  for  the  twenty  item 
posttest  was  0.72.  The  reliability  coefficients  are  the  same,  however  differences  exist  between  the  tests 
allowing  greater  confidence  in  comparing  the  two  tests. 
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An  ANOVA  model  was  used  in  a  2  (group)  X  2  (Time:  pretest  vs.  posttest)  analysis  of  variance  with 
repeated  measures  on  the  last  variable  (see  Table  2).  Analysis  revealed  no  difference  between  the 
experimental  group  and  the  control  group  overall:  F(l,44)  =  .07,  p  =  .79.  Analysis  revealed  a  significant 
difference  overall  within  the  groups  between  the  pretest  and  the  posttest:  F(l,44)  =  15.91,  p  <  .05. 
Analysis  revealed  no  interactive  effect  between  the  groups:  F{1,44)  =  .10,  p  =  .75  (the  interactive  effect  is 
shown  in  figure  2). 


Table  2. 

ANOVA  of  group  X  time  (pretest  vs.  posttesi) 


Source 

df 

MS 

F 

Between  Subjects 

Group 

1 

1.71 

.07 

Error 

44 

24.61 

Within  Subjects 

Time 

1 

31.86 

15.91* 

Group  X  Time 

1 

.21 

.10 

Error  . 

44 

2.00 

*_p_  <  .05 


Figure  2. 


Comparison  of  score  differences  in 
identification  of  harmonic  voicing 


Where: 

"■"  Experimental 
■D-  Control 


Pretest 


Posttest 


a 
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An  apriori  hypothesis  tested  using  a  Scheffe"  contrast  showed  the  experimental  group  differed  between  the 
pretest  and  the  posttest:  F(l,44)  =  2.98,  p  <  .05;  the  control  group  effect  across  the  pretest  and  posttest 
was  negligible:  F(l,44)  =  2.65,  p  =  .11.  Winer  (1962,  p.  85)  observed  "The  apriori  type  is  always 
justified  whether  or  not  the  overall  F  is  significant". 

Conclusion  regarding  the  First  Sub-problem. 

The  between-group-effect  was  negligible  (p>.05).,  indicating  that  no  differences  existed  between  the  control 
group  and  the  experimental  group  overall  (collapsed  across  the  tests),  however  the  significant  within-group- 
effect  (tj<05)  between  the  pretest  and  the  posttest  showed  that  there  is  a  difference  in  the  overall 
achievement  scores  between  the  two  tests  (collapsed  across  the  groups).  Where  the  ANOVA  model  uses  the 
overall  scores  for  analysis,  the  Scheffe  contrast  does  not  collapse  across  the  groups  and  therefore  can 
attribute  the  difference  noted  in  the  ANOVA  to  a  specific  source:  in  this  case,  a  difference  when  contrasting 
the  pretest  and  posttest  scores  of  each  group  The  contrast  showed  a  difference  between  the  pretest  and 
posttest  scores  of  the  experimental  group  but  not  the  control  group.  Relative  to  the  first  sub-problem, 
interactive  audio  resulted  in  better  skill  development  in  identifying  harmonic  voicing  than  non-interactive 
audio. 

Results  for  the  Second  Sub-problem 

To  determine  student  attitude  toward  CBI  in  music,  a  post-treatment  attitudinal  measurement  was  taken 
using  a  Likert-type  survey.  The  category  results  are  raw  scores  in  five  categories,  each  set  corresponding  to 
one  individual  from  either  the  experimental  or  control  group.  Category  A  deals  with  instructional  strategy 
scored  out  of  seventy.  Category  B  deals  with  the  personalization  of  CBI  scored  out  of  thirty.  Category  C 
deals  with  individual  reaction  scored  out  of  sixty  five.  Category  D  deals  with  interest  in  the  subject  matter 
scored  out  of  twenty.  Category  E  deals  with  the  technical  operation  of  the  equipment  scored  out  of  twenty- 
five.  The  total  is  the  sum  of  all  categories  and  represents  an  overall  positive  attitude  toward  CBI  in  aural 
skills  scored  out  of  two-hundred-ten.  Table  3  and  figure  3  summarize  the  data  for  the  second  sub-problem. 

Table  3. 


Positive  attitudes  toward  CBI  in  aural  skills 


Mean  Values : 

n 

Cat 

A 

Cat 

B 

Cat 

C 

Cat 

D 

Cat 

E 

Total 

Experimental 

22 

46 

6 

20 

7 

43 

.  0 

14 

6 

16 

.  4 

141 

.  4 

Control 

24 

47 

3 

 1 

17 

1 

43 

.  0 

14 

2 

16 

.  0 

137 

.  7 

All  Subjects 

46 

47 

o 

18  .  8 

43 

.  0 

14 

4 

16 

.  2 

139 

.  4 

SD  Values : 

n 

Cat 

A 

Cat 

B 

Cat 

C 

Cat 

D 

Cat 

E 

Total 

Experimental 

22 

5  . 

6 

2  . 

5 

5  . 

3 

2  . 

4 

2  .  8 

14  . 

6 

Control 

24 

6  . 

5 

4  . 

0 

7  . 

0 

2  . 

5 

3  . 

3 

19  . 

4 

All  Subject 

;  46 

6  . 

0 

3  .  8 

6  . 

1 

2  . 

5 

3  . 

0 

17  . 

2 

where:  Cat.A-instructional  strategy,  Cat.B-personalization  of  CAI,  Cat.C-individual 
reaction,  Cat.D-interest  in  subject  matter,  Cat.E-technical  operation  of  equipment 


Figure  3 

Positive  attitudes  toward  CBI  in  aural  skills 


Where: 

Experimental 
Control 
Overall 

Cat  A  -  instructional  strategy 
personalization  of  CAI 
individual  reaction 
interest  in  subject  matter 
technical  operation  of  equipment 


Cat  A  CatB  CatC  CatD  CatE  Cat  A  Cat  B  CatC  CatD  CatE 
Mean  Mean  Mean  Mean  Mean    SD      SD      SD      SD  SD 


Reliability  coefficient  alpha  for  the  forty-two  item  attitude  survey  was  0.92.  In  addition,  Cronbach's  alpha, 
which  accounts  for  relative  weights  between  the  five  sub-categories,  was  0.81.  In  each  case,  alpha  is  higher 
than  the  accepted  minimum  value  of  0.65  when  working  with  a  group  of  scores  (Frisbie,  1988).  These 
figures  indicate  high  internal  consistency  and  reliability  in  the  survey.  The  data  indicated  a  small  difference 
in  opinion  toward  CBI  in  category  B;  a  very  small  difference  in  categories  A,  D  and  E;  and  category  C 
showed  no  difference.  An  ANOVA  model  was  used  to  determine  if  any  of  these  observed  effects  were 
significant.  In  order  to  implement  the  ANOVA,  the  raw  scores  were  first  translated  into  z-scores  (see  Table 
4). 

Table  4. 


Standardized  scores  of  positive  attitudes  toward  CBI  in  aural  skills 


z-scores : 

n 

Cat  A 

Cat  B 

Cat  C 

Cat  D 

Cat  E 

Mean 

Experimental 

22 

-  .  06 

.  50 

.  00 

.09 

.  06 

.  12 

Control 

24 

.  06 

-  .  46 

-  .  00 

-  .  08 

-  .  05 

-  .  11 

Mean 

46 

.  00 

.00 

.00 

.  00 

.  00 

.00 

where:  Cat  A-instructional  strategy,  Cat  B-personalization  of  CAI,  Cat  C-individual 
reaction,  Cat  D-interest  in  subject  matter,  Cat  E-technical  operation  of  equipment 
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Figure  4. 

Comparison  of  standard  score  differences  of  positive  attitudes 


The  data  conversion  to  z-scores  results  in  a  non-legitimate  main  effect  within  group  (the  mean  in  each 
category  is  zero).  For  the  purposes  of  this  study,  the  within  group  effect  was  not  important,  however,  the 
between  group  effect  was  important.  The  analysis  revealed  no  difference  between  the  experimental  group 
and  the  control  group  (p>.05).  The  analysis  showed  a  significant  interactive  effect:  F(4,176)  =  4.83,  p  < 
.05. 

An  analysis  of  contrasts  was  performed  (Scheffe)  in  order  to  attempt  to  identify  the  source  of  the  interaction. 
The  analysis  showed  no  significant  contrasts  between  any  of  the  categories  (p>.05). 
Conclusion  regarding  the  Second  Sub-problem. 

The  between  group  effect  in  the  second  sub-problem  was  negligible,  indicating  no  overall  difference  between 
the  experimental  group  and  the  control  group  (see  the  mean  slope  in  figure  3).  A  significant  interaction 
was  noted;  however,  given  that  the  subsequent  contrasts  revealed  no  differences  within  individual  groups, 
the  interaction  was  considered  marginal.  The  z-scores  for  categories  A,  C,  D  and  E  all  approached  zero, 
however  the  z-scores  for  category  B  were  anomalous.  Even  though  the  contrasts  showed  no  difference,  the 
anomalous  scores  in  category  B  warrant  discussion  (see  figure  4). 

The  z-scores  in  category  B  represent  the  participant's  reaction  to  the  personalization  of  the  instructional 
material.  It  is  interesting  to  note  that  the  group  using  the  interactive  audio  responded  more  positively 
toward  the  personalization  of  the  program  than  did  the  group  using  non-interactive  audio.  This  may  indicate 
that  interactivity  results  in  a  perceived  increase  in  user-friendliness  or  personalization  in  CB1.  This 
speculation  is  based  only  the  anomalous  slope  of  category  B,  it  should  be  noted  that  statistically  this  score 
falls  within  the  probability  limits  of  non-significance. 

Summary  of  Results 

The  two  sub-problems  were  handled  independently  using  statistical  analysis  to  derive  greater  meaning  from 
the  raw  data.  The  analysis  was  done  using  a  probability  level  of  .05  to  determine  significance.  Based  on 
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test  results  which  were  reliable,  the  data  for  the  first  sub-problem  showed  significant  effects,  indicating  that 
CBI  with  interactive  audio  resulted  in  better  skill  development  at  identifying  harmonic  voicing  than  did  CBI 
with  non-interactive  audio.  Based  on  test  results  which  had  high  reliability,  the  data  for  the  second  sub- 
problem  showed  no  significant  effects,  indicating  that  no  differences  regarding  attitude  toward  CBI  in  aural 
skills  were  present  between  the  two  groups. 

Conclusions  and  Recommendations 

Conclusions 

The  results  of  this  study  are  limited  to  CBI  in  identification  of  harmonic  voicing  in  aural  skills  and  may 
only  be  generalized  to  similar  populations  of  music  and  music  education  students  in  Canada.  The  attitude 
of  the  experimental  grojup  did  not  differ  significantly  from  the  attitude  of  the  control  group;  however,  all 
participants  volunteered  for  the  study  without  coercion  and  the  sample,  therefore,  may  have  had  a 
predisposed  positive  opinion  about  computers  in  music  education.  This  could  have  been  a  factor  in  their 
decision  to  volunteer,  and  also  a  factor  which  may  have  affected  the  attitude  survey  -  possibly  resulting  in  a 
general  positive  attitude  towards  CBI  in  aural  skills.  Confidence  in  any  generalizations  regarding  attitude  in 
this  study  is  limited. 

With  regard  to  achievement,  the  skill  development  in  the  experimental  group  was  greater  than  that  of  the 
control  group:  this  supports  the  use  of  CBI  with  interactivity  in  aural  skills.  Confidence  in  the 
measurement  of  achievement  is  supported  knowing  that  the  sample  was  homogenous  with  regard  to  attitude 
towards  CBI  in  aural  skills:  discrepant  attitudes  between  the  groups  can  be  ruled  out  as  a  possible  source 
for  differing  achievement  between  the  groups.  The  measurement  results  show  acceptable  to  high  reliability 
coefficients. 

Some  conditions  in  the  study  may  help  to  explain  why  no  significant  differences  were  found  between  the 
attitudes  of  the  experimental  and  control  groups.  With  the  exception  of  interactive  control  of  harmonic 
voices  (the  basis  for  the  study),  the  CBI  modules  were  identical.  The  participants  were  involved  in  a  style 
of  aural  skills  training  which  was  relatively  new  and  unused  in  regular  aural  skills  classes  and  as  a  result  the 
overall  novelty  of  this  instruction  may  have  overshadowed  any  attitudinal  differences  attributable  to  the 
interactivity  and  may  have  provoked  positive  attitudes  in  all  users.  Previous  studies  indicate  that  students 
show  more  positive  attitudes  toward  CBI  in  aural  skills  than  toward  traditional  classroom  aural  skills 
training.  It  is  likely  that  when  both  groups  used  CBI  in  aural  skills,  the  differences  in  interactivity  while 
effecting  achievement  may  not  have  effected  attitude. 

Each  participant  was  active  in  the  study  for  approximately  one  hour,  in  which  time,  the  pretest,  treatment, 
posttest  and  attitude  modules  were  completed.  Participants  may  have  required  a  longer  exposure  to  CBI  in 
order  to  offer  accurate  statements  of  attitude. 

Similarly,  several  factors  warrant  discussion  which  may  help  to  explain  other  reasons  for  the  differences 
found  in  achievement.  One  hour  may  not  have  allowed  enough  time  to  accurately  measure  learning  in  a 
subject  area  which  the  participants  have  spent  years  studying.  Claims  regarding  effectiveness  of 
interactivity  may  require  longer  exposure  to  treatment  in  order  to  be  valid.  Although  this  study  accurately 
shows  differences  in  achievement  between  groups  using  interactive  versus  non-interactive  audio,  greater 
validity  may  be  gained  by  applying  the  treatment  over  periods  of  time  which  are  proportional  to  the  time 
students  spend  learning  aural  skills  in  regular  classes  at  university.  Pretest  contamination  may  have  been  a 
factor  which  resulted  in  increased  scores  on  the  posttest.  The  analysis  showed  no  overall  difference  between 
the  groups  but  did  show  a  difference  within  the  groups,  possibly  attributable  to  exposure  to  the  pretest, 
however,  because  the  participants  were  randomly  assigned  to  either  the  control  group  or  the  experimental 
group  and  both  groups  took  the  same  pretest,  it  is  assumed  that  any  pretest  contamination  would  be  equal 
for  both  groups,  leaving  any  differences  attributable  to  the  treatment  differences  between  the  groups.  The 
results  of  the  contrast  analysis  support  this  by  attributing  the  differences  to  the  treatment,  specifically  to  the 
interactivity  built  into  the  program  used  by  the  experimental  group. 

These  results  support  the  findings  by  Woodrulff  and  Heeler  (1990)  which  indicate  that  interactive  audio 
results  in  higher  achievement  by  music  appreciation  students;  and,  also  support  Ashley's  conclusions 
(1989)  that  interactive  audio  is  effective  in  increasing  the  listening  skills  of  music  students.  Support  for 
CBI  in  aural  skills  and  continued  research  in  the  area  is  consistent  with  many  other  studies  (Arenson,  1982; 
Bailey,  1989;  Canelos,  Murphy,  Blombach  &  Heck,  1980;  Dalby,  1989;  Greenfield  &  Codding,  1985; 
Hofstetter,  1978,  1979,  1980,  1981).  Technology  is  advancing  at  an  ever-increasing  rate  and  the  tools 
which  become  available  to  the  aural  skills  student  may  advance  to  the  point  where  comparisons  with  older 
technology  are  no  longer  valid:  this  only  strengthens  the  need  for  ongoing  research  in  the  area. 
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Recommendations 


Two  limiting  factors  in  this  study  could  be  eliminated  in  future  studies.  The  sample  size  of  forty-six 
participants  is  small:  results  using  larger  samples  might  be  generalized  to  larger,  more  diverse  populations. 
Also,  the  treatment  time  in  this  study  was  short:  each  student  spent  approximately  one  hour  at  the 
computer.  The  close  proximity  in  time  between  the  pretest  and  the  posttest  may  have  affected  the  test 
results.  A  study  which  is  structured  over  a  longer  period  (such  as  a  semester)  would  likely  allow  greater 
confidence  in  the  results. 

This  study  used  synthesized  sounds  from  tone  generators  within  the  Macintosh  computer.  These  sounds  are 
accurate  with  regard  to  pitch  and  also  include  specific  overtones  which  create  a  sound  more  similar  to  a 
natural  sound  than  a  sine  wave;  however,  truly  natural  sounds  can  only  be  achieved  through  sampling 
technology,  or  very  sophisticated  synthesis.  A  study  using  CBI  in  aural  skills  could  be  designed  to 
determine  the  effect  of  natural  (sampled)  sounds  versus  synthesized  sounds  on  student  achievement  and 
attitude.  A  similar  study  could  determine  if  music  students  respond  better  in  aural  skills  to  the  natural 
sound  of  their  major  instrument  than  to  other  instruments.  For  example,  can  a  trumpet  major  identify 
intervals  played  on  a  trumpet  better  than  intervals  played  on  a  piano? 

CBI  has  great  potential  for  accommodating  further  research  in  aural  skills  with  new  advancements  in 
hardware,  software,  MIDI  and  sampling  technologies. 


References 

Arenson,  M.A.  &  Hofstetter,  F.T  (1983).  High-tech  models  for  music  learning:  The  GUIDO  system  and 

the  PLATO  project.  Music  Educators  Journal.  69(5),  46-51. 
Arenson,  M.A.  (1982).  The  effect  of  a  competency-based  computer  program  on  the  learning  of  fundamental 

skills  in  a  music  theory  course  for  non-music  majors.  Journal  of  Computer-Based  Instruction. 

9(2),  55-88. 

Ashley,  R.D.  (1989).  Redesigning  the  content  and  sequence  of  instruction  in  music  theory:  final  report  to 
fund  for  the  improvement  of  post  secondary  education.  Evanston,  Illinois:  Northwestern 
University.  (ERIC  Document  Reproduction  Service  No.  ED  340  630) 

Bailey,  D.L.  (1989).  The  effects  of  computer-based  instruction  on  achievement  of  four,  five  and  six-year- 
old  children  in  the  Yamaha  Music  Education  System  Primary  One  Course.  Dissertation  Abstracts 
International.  51(3),  777A. 

Bloom,  B.S.  (1984).  The  2  sigma  problem:  The  search  for  methods  of  group  instruction  as  effective  as 
one-to-one  tutoring.  Educational  Researcher,  13(4),  4-16. 

Canelos,  J.J.,  Murphy,  B.A.,  Blombach,  A.K.,  &  Heck,  W.C.  (1980).  Evaluation  of  three  types  of 

instructional  strategy  for  learner  acquisition  of  intervals.  Journal  of  Research  in  Music  Education, 
28_,  243-249. 

Clarkson,  A.E.  &  Pegley,  K.  (1991).  An  assessment  of  a  technology  in  music  programme.  (Technical 

report  9 1  -2,  revised  edition.)  Centre  for  the  Study  of  Computers  in  Education,  York  University. 

(ERIC  Document  Reproduction  Service  No.  ED  341  364) 
Dalby,  B.F.  (1989).  A  computer-based  training  program  for  the  development  of  harmonic  intonation 

discrimination  skill.  Dissertation  Abstracts  International.  5H(7),  241  A. 
Frisbic,  D.A.  (1988).  Reliability  of  scores  from  teacher-made  tests.  Educational  Measurement,  7(1  )■  25- 

32. 

Gillespie,  F.,  &  Placek,  R.  W.  (1991).  Technology  for  teaching.  Music  Educators  Journal.  77(5).  53-54. 
Greenfield,  D.G  &  Codding,  P.A.  (1985).  Competency-based  vs.  linear  computer  instruction  of  music 

fundamentals.  Journal  of  Computer-Based  Instruction.  12(4).  108-110. 
Gross,  D.  &  Griffin,  W.  (1982).  Implementation  and  evaluation  of  a  computer-assisted  course  in  musical 

aural  skills.  AEDS  Journal.  15(3).  143-150. 
Hoffmann,  J.A.  (1991).  Computer-aided  collaborative  music  instruction.  Harvard  Educational  Review, 

6_L(3),  270-278. 

Hofstetter,  F.T.  (1975).  GUIDO:  An  interactive  computer-based  system  for  improvement  of  instruction 
and  research  in  ear  training.  Journal  of  Computer-Based  Instruction,  1,  100-106. 


a 

ERIC 


671^0?  J 


Hofstetter,  F.T.  (1978).  Computer-based  recognition  of  perceptual  patterns  in  harmonic  dictation  exercises. 

Journal  of  Resean    in  Music  Education.  26,  111-119. 
Hofstetter,  F.T.  (1979).  Evaluation  of  a  competency-based  approach  to  teaching  aural  interval 

identification.  Journal  of  Research  in  Music  Education,  27,  201-213. 
Hofstetter,  F.T.  (1980).  Computer-based  recognition  of  perceptual  patterns  in  chord  quality  dictation 

exercises.  Journal  of  Research  in  Music  Education.  2&,  83-91. 
Hofstetter,  F.T.  (1981).  Computer-based  aural  training:  The  GUIDO  system.  Journal  of  Computer-Based 

Instruction,  2(3),  84-92. 

Hofstetter,  F.T.  (1981).  Computer-based  recognition  of  perceptual  patterns  and  learning  styles  in  rhythmic 

dictation  exercises.  Journal  of  Research  in  Music  Education,  22(4),  265-277. 
Humphreys,  J.T.  (1986).  Measurement,  prediction,  and  training  of  harmonic  audiation  and  performance 

skills.  Journal  of  Research  in  Music  Education.  34(3),  192-199. 
Humphries,  J.  A.  (1980).  The  effects  of  computer-assisted  aural  drill  time  on  achievement  in  musical 

interval  identification.  Journal  of  Computer-Based  Instruction.  6(3),  91-98. 
Kirk,  R.E.  (1968).  Fundamental  assumptions  in  analysis  of  variance.  Experimental  design:  Procedures  for 

the  behavioral  sciences,  (pp.  60-63).  Belmont,  California:  Brooks/Cole. 
Kulik  J.A.,  Kulik,  C.C.,  &  Cohen,  P.A.  (1980).  Effectiveness  of  computer-based  college  teaching:  A 

meta-analysis  of  findings.  Review  of  Educational  Research,  50.(4),  525-544. 
Kuzmich,  J.,  Jr.  (1989).  New  styles,  new  technologies,  new  possibilities  in  jazz.  Music  Educators 

Journal.  76(3),  41-46. 

Madsen,  C.K.  &  Geringer,  J.M.  (1990).  Differential  patterns  of  music  listening:  Focus  of  attention  of 
musicians  versus  non  musicians.  Bulletin  of  the  Council  for  Research  in  Music  Education,  1Q5, 
45-57. 

Newcomb,  S.R.  ( 1988).  Computer-based  arts  instruction:  How  are  we  doing?  Design  for  Arts  in 
Education.  22(5),  46-49. 

Newcomb,  S.R.,  Weage,  B.K.,  &  Spencer,  P.  (1981).  MEDICI:  Tutorial  in  melodic  dictation.  Journal  of 

Computer-Based  Instruction.  2(3),  63-69. 
Pembrook,  R.G.  (1986).  Some  implications  of  students'  attitudes  toward  a  computer-based  melodic 

dictation  program.  Journal  of  Research  in  Music  Education.  34(2),  121-133. 
Pogonowski,  L.  (1989).  Critical  thinking  and  music  listening.  Music  Educators  Journal.  76(1),  35-38. 
Taylor,  J. A.  (1982).  The  MEDICI  melodic  dictation  computer  program:  Its  design,  management,  and 

effectiveness  as  compared  to  classroom  melodic  dictation.  Journal  of  Computer-Based  Instruction. 


Taylor,  J. A.  (1988).  Computers  in  music  and  music  instruction:  The  joys  of  hardware  and  the  woes  of 

software.  Design  for  Arts  in  Education.  S2(5),  50-55. 
Upitis,  R.  (1982).  A  computer-assisted  instruction  approach  to  music  for  iunior-aae  children:  Using  ALF 
for  teaching  music.  (Paper  presented  at  the  International  Computer  Music  Conference,  Venice, 
Italy,  September  1982).  Queen's  University,  Kingston,  Ontario.  (ERIC  Document  Reproduction 
Service  No.  ED  225  650) 
Weintraub,  D.  (1991).  Improving  retention  in  music  fundamentals  through  the  use  of  computer  based 
instruction.  (Ed  D  Practicum).  Nova  University.  (ERIC  Document  Reproduction  Service  No. 
ED  336  072) 

Woodrulff,  E.,  &  Heeler,  P.  (1990).  A  study  of  the  use  of  intractive  videodisc  technology  to  present  aural 
tests  to  college  music  appreciation  students.  (National  Educational  Computing  Conference 
Proceedings).  Nashville,  TN.  (ERIC  Document  Reproduction  Service  No.  ED  329  232). 

Winer,  B.  J.  (1962).  Statistical  principles  in  experimental  design.  NY:  McGraw-Hill. 

Woodrulff,  E.,  &  Heeler,  P.  (1991).  A  computer-based  tutorial  system  to  develop  listening  skills  using 

CD-ROM.  (Paper  presented  at  the  Joint  conference  of  the  Educational  Computing  Organization  of 
Ontario  and  the  International  Conference  on  Technology  and  Education).  Toronto,  Ontario. 
(ERIC  Documeni  Reproduction  Service  No.  ED  346  840) 


2(2),  64-73. 


672 


ERIC 


Appendix 


Description  of  the  Program 

The  four  parts  of  the  computer  program  are  described  below.  Although  the  modules  are  each  self-contained 
executable  programs,  they  are  transparently  linked  together  in  one  seamless  presentation. 

The  first  part  of  the  program  is  a  pretest  of  abilities  in  aural  skills  (see  Appendix  B  for  a  hard  copy  of  the 
measurement  items).  Using  triad  samples  prepared  for  audible  computer  del  ivery,  the  user  identifies  the 
quality  of  each  triad  as  major,  minor  or  diminished.  To  facilitate  a  two-group  experimental  design,  there  arc 
two  versions  of  this  module:  Module-One  which  links  to  Module-Two  (using  non-interactive  audio)  and 
Module-One-i  which  links  to  Module-Two-i  (using  interactive  audio) 

The  second  part  of  the  program  is  a  lesson  in  identification  of  triad  quality  (major,  minor  and  diminished). 
Both  versions  of  this  module  link  to  Module-Three. 

The  third  part  of  the  program  is  a  posttest  of  abilities  in  aural  skill,  similar  to  the  pretest  in  construction 
and  content  (see  Appendix  B  for  a  hard  copy  of  the  measurement  items).  Although  the  questions  in  the 
posttest  are  identical  in  structure  to  the  questions  in  the  pretest,  the  posttest  uses  different  audible  samples. 
This  module  links  to  Module-Four. 

Finally,  the  fourth  part  is  a  Likert-type  survey  of  opinion  toward  CBI  in  aural  skills  (see  Appendix  B  for  a 
hard  copy  of  the  measurement  items).  The  attitude  survey  is  based  on  an  instrument  to  measure  opinion 
toward  CBI  from  M.  Szabo  (persona!  communication,  March  24,  1992).  The  instrument  categorizes 
student  responses  into  five-sub-areas  and  a  total.  Each  score  in  the  sub-areas  represents  the  relative  positive 
attitude  the  student  has  toward  CBI  in  aural  skills. 

The  modules  write  all  participant  data  to  unique  external  files  (see  Appendix  C  for  a  file  listing  of  the  raw 
data  totals  gathered  in  this  study).  A  file  of  all  responses  and  calculated  totals  called  "MassData"  is  created 
in  the  same  location  as  the  RunAPM  file  which  is  used  to  run  the  modules.  The  record  layout  of  the  file  is 
shown  below. 


Column  1       -  group  ID(1  or  2) 
Column  2-2 1   -  pretest  answers  ( 1 ,  2,  3,  or  4) 
Column  22-41  -  posttest  answers  (1,  2,  3,  or  4) 
Column  42-83  -  attitude  responses  (1,  2,  3,  4,  or  5) 

Following  column  83  is  a  TAB  delimited  series  of  numbers: 

Pretest  total  out  of  20 

Listen  again  number  of  times  the  user  asked  to  re-hear  a  music  sample 

Volume  control  number  of  times  the  user  adjusted  the  volumes  of  voices 

Posttest  total  out  of  20 

Attitude  category  A  total  out  of  70 

Attitude  category  B  total  out  of  30 

Attitude  category  C  total  out  of  65 

Attitude  category  D  total  out  of  20 

Attitude  category  E  total  out  of  25 

Attitude  total  total  out  of  210 


In  addition  a  file  called  "Comment  File"  is  created  in  the  same  location  and  contains  any  user  comments 
which  were  anonymously  logged  during  the  session. 


MassData  record  layout 
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The  program  was  validated  by  content  experts  in  aural  skills.  The  supervisor  of  aural  skills  in  the  music 
department  provided  content  information  during  the  construction  of  the  program,  made  revisions  and 
subsequently  approved  the  program  following  completion.  In  addition,  a  music  education  faculty  member 
made  revisions  to  the  program  during  development  and  subsequently  offered  approval.  The  program  was 
also  validated  by  an  instructional  design  expert  who  made  revisions  and  provided  subsequent  approval. 


Music  Terminology 

A  brief  explanation  of  the  musical  terminology  and  symbols  used  is  included  here  in  order  to  add 
clarification  to  any  subsequent  musical  references.  During  the  computer  session  the  students  were  asked  to 
identify  the  chord  quality  of  audible  triads.  Only  major,  minor  and  diminished  chord  qualities  were  used.  It 
is  important  to  recognize  that  musical  theory  is  not  the  focus  of  the  study,  but  rather  the  aural 
discrimination  of  differences  in  the  chord  qualities.  Therefore,  for  the  purposes  of  this  document,  the  theory 
is  explained  in  a  manner  suitable  for  understanding  the  aural  significance  of  the  task  and  not  in  a  manner 
typically  found  in  standard  musical  references. 

A  triad  is  a  combination  of  three  notes,  typically  spaced  in  intervals  of  thirds,  which  form  a  chord.  There 
are  two  types  of  thirds:  major  and  minor,  where  a  major  third  is  an  interval  of  four  semitones  and  a  minor 
third  is  an  interval  of  three  semitones.  The  entire  harmonic  structure  of  music  in  the  western  world  is  based 
on  a  scale  of  twelve  semitones  where  a  semitone  is  the  distance  (in  frequency)  between  notes.  The  figure 
below  shows  the  repeating  scale  of  semitones  on  the  familiar  piano  keyboard. 

Twelve-tone  scale  on  piano 
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A  minor  third  would  be  an  interval  from  note  1  to  note  4  (a  difference  of  three  semitones),  and  a  major  third 
would  be  an  interval  from  note  1  to  note  5  (a  difference  of  four  semitones).  Similarly,  another  minor  third 
would  be  an  interval  from  note  5  to  note  8,  and  a  major  third  from  note  5  to  note  9,  and  so  on.  Each 
interval  is  relative  to  the  lower  note. 

A  major  triad  is  formed  from  a  major  third  and  a  minor  third.  For  example,  a  major  triad  could  be  formed 
from  note  1  and  note  5  (major  third)  and  note  8  (a  minor  third  from  note  5). 

A  minor  triad  is  formed  from  a  minor  third  and  a  major  third.  For  example,  a  minor  triad  could  be  formed 
from  note  1  and  note  4  (minor  third)  and  note  8  (a  major  third  from  note  4). 

A  diminished  triad  is  formed  from  two  minor  thirds.  For  example,  a  diminished  triad  could  be  formed  from 
note  1  and  note  4  (minor  third)  and  note  7  (a  minor  third  from  note  4). 

Similarly,  triads  can  be  formed  from  any  starting  note  as  long  as  the  intervals  remain  relative.  For 
example,  a  major  triad  could  be  formed  starting  on  note  3  and  would  consist  of  note  3,  note  7,  and  note  10. 
A  minor  triad  could  also  be  formed  starting  on  note  3  and  would  consist  of  note  3,  note  6  and  note  10.  A 
diminished  triad  could  be  formed  starting  on  note  3  and  would  consist  of  note  3,  note  6  and  note  9. 
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The  students  are  required  to  differentiate  aurally  between  these  three  types  of  qualities  of  triads 
differences  are  visually  summarized  in  the  figures  below. 


The 


Major,  minor  and  diminished  triads  on  piano 


Minor  Diminished 
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Abstract 


This  study  investigated  the  effects  if  augmenting  and  deleting  elaborations  in  an  existing  self- 
instructional  text  for  a  micro-computer  database  application.  A  total  of  60  undergraduate  students  were 
randomly  assigned  to  the  original,  elaborated  or  unelaborated  text  versions.  Students  completed  six  tasks 
using  one  of  the  three  instructional  texts  and  then  completed  a  six-item  attitude  survey.  The  absence  of  a 
significant  difference  in  the  task  performance  under  the  three  text  versions  by  a  repeated  measures  ANOVA 
suggests  that  not  all  people  require  elaborations  to  develop  a  good  understanding  of  procedural  learning. 


The  Effects  of  Elaboration  on  Self-Learning  Procedures  From  Text 

Reading  from  text  is  one  way  in  which  people  acquire  knowledge  or  skills.  An  effective 
instructional  text  makes  learners  become  active  participants  in  the  learning  process  rather  than  passive 
absorbers  of  information.  In  an  age  of  independent,  self-directed  study,  learning  from  texts  is  a  common 
strategy,  particularly  in  the  computer  industry.  With  the  spread  of  personal  computer  systems,  the  issue  of 
how  to  design  effective  self-instructional  texts  that  facilitate  the  acquisition  of  knowledge  and  skills  has 
become  a  very  important  one. 

It  is  not  uncommon  to  hear  students  complain  about  technical  textbooks  such  as  computer 
manuals.  Obviously,  if  an  instructional  text  designed  for  self-directed  learning  is  too  difficult  to  read  or  has 
confusing  terminology,  learners  will  become  frustrated,  and  the  negative  experience  would  lower  their 
motivation  for  learning.  Pepper  (1981)  reported  that  students  were  looking  for  easy-to-read  texts  supported 
by  many  examples.  Unfortunately,  many  computer  textbooks  and  manuals  don't  use  examples. 

Documentation 

Duin  ( 1990)  argued  that  computer  documentation  should  center  on  the  learner's  goals,  instead  of  on 
the  system  or  on  the  concepts  that  are  being  taught.  The  learner's  primary  goal  of  reading  computer 
manuals  is  to  learn  how  to  use  the  computer  programs  (skill  learning),  not  only  to  know  what  the 
programs  are  (fact  learning).  In  cognitive  science  literature,  a  similar  distinction  between  fact  learning  and 
skill  learning  has  been  considered  under  the  label  of  declarative  versus  procedural  learning.  Since  learning  to 
use  a  computer  is  a  kind  of  procedural  (skill)  learning,  learners  want  to  extract  information  for  immediate 
action  rather  than  absorb  knowledge  for  future  recall. 

Developers  of  computer  documentation  should  also  consider  the  cost  of  ineffective  instructional 
texts.  When  instruction  is  inefficient,  the  costs  incurred  by  both  learners  and  business  are  extremely  high 
(Carroll,  1990).  Thus,  the  importance  of  designing  an  effective  instructional  text  that  facilitates  procedural 
learning  cannot  be  overemphasized. 

Text-Based  Elaboration 

One  variable  that  may  increase  the  effectiveness  of  learning  (particularly  self-directed  learning)  from 
text  is  elaboration.  According  to  Gagne  "Elaboration  is  the  process  of  adding  related  knowledge  to  the 
information  being  learned.  The  addition  can  be  a  logical  inference,  a  continuation,  an  example,  a  detail,  or 
anything  else  that  serves  to  connect  information"  (Gagne,  1985,  p.83).  A  common  goal  of  elaboration  is 
to  attempt  to  make  learning  more  meaningful  by  forming  a  relationship  between  the  new.  unfamiliar 
material  and  the  older,  already  learned  information.  There  has  been  ample  research  that  indicates 
elaborations  could  help  subjects  learn  and  remember  the  main  ideas  of  a  text,  because  elaborations  provide 
multiple  retrieval  routes  to  the  essential  information  by  creating  more  connections  to  the  learner's  prior 
knowledge  (Reder,  Charney,  &  Morgan,  1986).  Elaborations  also  provide  extra,  useful  information  for 
constructing  an  answer  (Reder,  1982).  That  is,  elaborations  can  lead  to  better  memory  by  increasing  the 
redundancy  of  interconnections  among  the  to-be-remembered  information,  and  by  imposing  an  organization 
on  the  information  that  can  be  used  to  guide  the  retrieval  process. 

Levin  (1988),  in  his  research  review  of  elaborated-based  learning  strategies,  indicated  that 
elaboration  can  facilitate  one's  memory  in  learning  because  people's  memory  for  nonmeaningful  or  arbitrary 
stimulus  materials  can  be  substantially  improved  through  the  addition  of  relevant  connections  or 
"mediators"  (p. 191).  Over  the  last  quarter  of  a  century,  there  has  been  a  great  deal  of  evidence  that  shows 
elaboration-based  strategies  to  facilitate  students'  performances  in  a  variety  of  school-learning  content  areas 
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and  tasks.  These  range  from  memory  for  simple  factual  material  to  the  processing  of  complex  prose 
passages  (see  Levin,  1988,  for  a  review).  In  addition,  elaborations  are  beneficial  for  individuals  with  diverse 
characteristics  and  at  all  levels  of  ability,  including  wide  variations  in  their  age,  ethnicity,  intelligence, 
academic  achievement,  and  specific  abilities  and  aptitudes. 

Reder  and  colleagues  (1984,  1986)  studied  the  informational  content  of  manuals  and  the  role  of 
text-based  elaborations.  They  defined  text-based  elaborations  as  "any  information  that  supports,  clarifies,  or 
further  specifies  the  main  point  of  a  text,  including  examples,  details,  analogies,  restatements,  and 
deductions"  (Reder,  Charney,  &  Morgan,  1986,  p. 64).  Results  from  a  large  body  of  research  consistently 
show  that  text-based  elaboration  facilitates  learning  in  the  acquisition  and  recall  of  declarative  knowledge. 
In  certain  cases,  such  as  for  children  with  mental  retardation,  the  more  elaboration,  the  better  (Levin,  1988). 
The  reason  is  elaborations  that  relate  to  more  than  one  part  of  the  new  knowledge  are  more  effective  in 
enhancing  retrieval  than  are  elaborations  that  relate  to  only  one  part  of  the  new  information.  However, 
elaborations  do  not  benefit  all  performance  outcomes.  For  example,  Mayer  (1980)  pointed  out  that  the 
research  indicates  that  elaboration  in  the  form  of  artificially  constructed  verbal  learning  materials  increased 
learner  efficiency  and  recall. 

By  contrast,  elaborations  can  also  interfere  with  recall.  There  is  ample  evidence  for  the  existence 
of  retrieval  interference  both  on  recall  and  on  response  times  to  verification  (Reder,  Charney,  &  Morgan, 
1986).  Although  elaborations  help  people  remember  relational  information  as  well  as  specific  information 
about  the  key  concepts  involved  in  the  relationship,  the  results  of  Stein  et  al.  (1984)  showed  that 
elaborations  that  enhance  loth  relational  and  item-specific  information  do  not  require  the  activation  of 
associatively  related  concepts  that  learners  have  previously  acquired. 

Learner-Generated  vs  Author-Provided  Elaboration 

The  definition  by  Reder,  Charney,  &  Morgan  in  1986  includes  two  sources  of  elaborations:  ( I ) 
those  generated  by  learners  while  reading,  and  (2)  those  contained  within  the  text  provided  by  the  author. 
Ideally,  given  a  reading  text,  students  would  elaborate  on  new  information  by  thinking  of  related  ideas, 
examples,  images,  or  logical  inferences  as  they  read.  Thus,  learner-generated  elaborations  are  likely  to  be 
more  relevant  to  his  or  her  prior  knowledge  and  purpose  for  reading  the  text.  On  the  other  hand,  author- 
provided  elaborations,  such  as  examples,  details,  or  embellishments,  may  be  more  accurate  than  those 
generated  by  the  learner  because  the  author  is  more  knowledgeable  and  more  familiar  with  the  topic. 

Elaborations  either  generated  by  the  learner  or  provided  by  the  author  have  their  merits  and 
drawbacks.  For  example,  learner-generated  elaborations  have  been  found  in  several  research  studies  to 
facilitate  memory  and  retention.  By  elaborating  on  the  presented  material  with  their  relevant  prior 
knowledge,  the  subjects  in  these  studies  reviewed  by  Reder,  Charney,  and  Morgan  in  1986  show  better 
retention  of  the  material  and  better  understanding  of  it.  However,  Reder  et  al.(1986)  also  indicated  that  not 
all  author-provided  elaborations  supply  similar  benefits  on  retention  of  the  central  ideas  in  all  learning 
situations,  although  they  reported  that  both  sour-es  of  elaborations  do  facilitate  skill  performance. 
Moreover,  Levin  reviewed  several  studies  and  suggested  that  with  inefficient  learners,  it  is  better  for  an 
instructor  to  provide  elaborations  than  to  have  the  students  generate  them  on  their  own  (Levin,  1988). 
Carroll  (1984)  also  argued  that  learner-generated  elaboration  such  as  "thinking,"  according  to  their  prior 
knowledge,  can  often  lead  to  mistakes  while  they  are  acquiring  knowledge  and  skill  in  the  computer  domain. 
Therefore,  in  some  situations  learner-generated  elaborations  may  be  most  effective,  while  in  others  author- 
provided  elaborations  may  be  best. 

A  number  of  factors  contribute  to  which  type  of  elaboration  is  most  appropriate  for  a  given 
situation.  These  include  individual  differences  such  as  prior  knowledge,  age,  ability,  and  the  learner's  own 
performance  objective.  Although  Weinstein  (1982)  provided  evidence  that  students  can  be  trained  to  use 
elaboration  to  enhance  learning,  it  remains  to  be  determined  whether  generalized  skills  can  be  taught  that 
would  enable  students  to  generate  their  own  elabcrators  for  a  variety  of  procedural  learning  tasks. 

Different  Forms  of  Elaborations 

Considerable  research  has  been  done  in  the  area  of  author-provided  elaborations  to  examine  the 
effects  of  different  forms  of  elaborations  including  examples,  details,  and  embellishments  on  fact  as  well  as 
on  skill  learning.  In  fact,  not  all  research  on  author-provided  elaborations  has  found  to  impair  both  kinds  of 
learning.  Generally  speaking,  the  research  suggests  that  author-provided  elaborations  may  facilitate  memory 
performance  if  they  are  constructed  in  view  of  the  specific  processes  operating  during  testing.  For  example, 
an  embellishing  detail  is  expected  to  facilitate  access  to  a  super  ordinate  major  idea  during  free  recall  while  it 
is  accessible  itself  and  possesses  sufficiently  strong  relations  to  the  relevant  major  idea  (van  Dam  & 
Brinkerink-Carlier,  1989).  Some  findings  regarding  the  use  of  author-provided  elaborations  have  shown  the 
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addition  of  embellishing  details  may  facilitate  future  access  to  the  embellished  main  ideas  of  a  text  during 
learning.  For  example,  Bradshaw  and  Anderson  (1982)  found  that  recall  performance  for  a  central  fact  was 
better  when  it  was  supported  by  elaborated  condition  than  when  it  was  studied  along  with  unelaborated 
condition.  In  addition,  recall  increases  as  the  number  of  related  details  increases  and  the  number  of  unrelated 
details  decreases  (Mohr,  Glover,  &  Ronning,  1984).  The  results  of  van  Dam  &  Brinkerink-Carlier's  study 
(1989)  indicated  that  embellishing  information  supplied  in  one  paragraph  may  actually  increase  the 
accessibility  of  another  paragraph  if  elaborations  allow  the  plot-irrelevant  information  to  be  related  to  the 
story.  However,  Reder  (1982)  concluded  that  author-provided  elaborations  appear  to  help  retention  of  the 
main  ideas  of  a  text  only  when  subjects  are  allowed  to  reconstruct  those  main  ideas  during  the  retention  test. 
Mandl  et  al.  (1984)  also  argued  that  elaborated  texts  facilitated  recall  and  comprehension  only  when  the 
subjects  were  very  familiar  with  the  topic;  otherwise,  elaborated  texts  performed  worse  than  unelaborated 
ones. 

These  findings  are  contrary  to  those  of  Reder  and  J.  R.  Anderson  (1980,  1982),  who  examined  the 
function  of  details  and  the  effects  of  embellishment  on  memory  for  the  main  points  of  a  text.  They  found, 
surprisingly,  that  subjects  performed  better  after  studying  an  abridged  or  summarized  version  of  the  original 
text  than  after  studying  a  full,  elaborated  text.  The  advantages  for  summaries  were  maintained  at  retention 
intervals  (from  20  minutes  to  12  months)  as  well  as  in  reaction  times  (subjects  answered  faster);  were 
superior  both  for  questions  directly  taken  from  the  text  and  for  inferences  that  required  the  subjects  to 
combine  facts  that  had  been  studied;  and  yielded  better  transfer,  too.  The  advantages  for  summaries  were 
found  under  a  variety  of  study  conditions  including  a  non  laboratory  setting  (Reder,  1982).  Performance 
was  superior  even  on  new  material  when  related  material  had  previously  been  studied  in  summary  form. 
Two  possible  causes  concluded  by  Reder  and  J.  R.  Anderson  (1982)  are  that  (1)  reading  summaries  allows 
the  subject  to  reread  the  main  points  at  spaced  intervals,  and  (2)  the  presence  of  details  distracts  the  subject's 
attention  away  from  the  critical  ideas  that  should  be  attended  to.  In  other  words,  details  hurt  retention  of 
central  ideas  within  a  text  when  the  study  time  is  fixed,  although  they  do  increase  interest  and  credibility. 
Moreover,  Reder  et  al.  (1986)  found  that  when  people  already  know  what  tasks  they  must  perform,  they 
benefit  very  little  from  seeing  examples  and  other  elaborations  in  the  text. 

Despite  the  assertion  that  author-provided  elaborations  may  impede  fact  learning  with  a  fixed  study 
time,  Reder,  Charney  and  Morgan  (1986)  argued  that  author-provided  elaborations  may  facilitate  the  domain 
of  skill  acquisition.  Elaborations  in  computer  documentation,  according  to  Duin  (1990),  may  expand  on 
what  the  basic  concepts  are,  when  these  concepts  are  relevant,  and  how  users  apply  these  to  their  immediate 
goals.  In  fact,  elaborations  seem  most  important  for  subjects  learning  to  select  and  execute  procedures 
correctly.  To  facilitate  skill-performance,  Reder  et  al.  (1986)  demonstrated  author-provided  elaborations  for 
a  personal  computer  that  provided  the  learner  with  explanations  and  concrete  illustrations  of  how  the  skill  is 
performed.  They  found  that  these  elaborations  helped  learners  solve  problems  more  quickly  than  minimal 
manuals  without  such  elaborations.  They  also  reported  that  only  syntactic  elaborations  that  illustrated  how 
correct  commands  should  look  helped  experienced  and  novice  computer  users  determine  exactly  how  to 
implement  a  procedure.  A  possible  reason  why  the  syntax  examples  facilitated  performance,  to  the 
exclusion  of  the  concept  elaborations,  is  that  perhaps  the  syntax  examples  more  closely  matched  what  the 
subject  needed  to  do. 

Consistent  with  this  point  of  view,  in  their  study  to  teach  writing  recursive  functions,  Pirolli  and 
J.  R.  Anderson  (1985)  found  that  the  subjects  whose  text  included  examples  and  explanations  of  correct 
syntax  performed  more  efficiently.  They  suggested  that  it  is  the  example,  which  is  used  to  illustrate  how  to 
do  the  task  rather  than  to  simply  clarify  what  happens  during  a  skill  learning  procedure,  helps  performance. 
Their  protocol  data  indicated  that  subjects  do  rely  heavily  on  examples  to  guide  their  solutions  to  novel  and 
difficult  problems.  Ross  (1984)  has  shown  that  performance  in  a  domain  of  new  skill  learning  is  affected 
by  prior  examples  when  these  examples  match  on  superficial  features.  Pepper  (1981)  reported  the 
surprising  findings  that  the  subjects  overwhelmingly  preferred  a  verbose  and  poorly  written  text 
(conversational-style)  over  a  concise  and  well  written  one  (outline-style)  because  they  attempted  to  look  for 
explanations  and  examples  for  help.  In  a  sense,  one  indicator  of  good  text  is  the  degree  to  which  writers 
elaborate  on  a  topic  with  explanations,  supporting  ideas,  and  examples  (Benton  &  Blohm,  1986).  Pepper 
(1981)  claimed  that  a  technical  text,  such  as  a  computer  documentation,  will  be  more  readable  and  more 
effective  by  avoiding  technical  jargon,  writing  in  a  clear  and  unpretentious  style,  and  incorporating  various 
graphical  and  textual  aids  into  the  body  of  the  text.  Thus,  a  new  experimental  chapter  was  written  by 
Pepper  according  to  his  guidelines  (see  Pepper,  1981,  p.  263  for  a  review),  and  the  results  showed  that  the 
subjects  who  read  the  new  chapter  performed  substantially  better  in  their  comprehension,  and  rated  the  new 
chapter  more  favorable  than  any  of  the  others.  Two  conclusions  that  Pepper  (1981)  suggested  for  the  field 
of  document  design  are:  ( 1 )  examples  should  be  used  as  a  tool  for  bridging  the  gap  between  the  expert  and 
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the  novice;  and  (2)  textbooks  should  be  written  for  the  student,  at  the  student's  level,  and  with  the  active 
guidance  of  the  student. 

Summing  up  the  findings  of  previous  research,  text-based  elaborations  facilitate  in  the  domain  of 
declarative  fact  learning  and  skill  performance.  To  some  degree,  a  useful  and  important  form  of  elaboration 
is  the  example,  particularly  for  helping  people  learn  a  procedure.  However,  several  studies  have  noted  that 
not  all  types  of  examples  provide  the  same  help.  Examples  that  were  found  to  facilitate  performance  are 
those  that  address  syntactic  information,  and  those  that  clarify  how  to  execute  commands  in  computer- 
related  procedures.  For  the  sake  of  learners'  needs  in  a  procedure  learning,  providing  concrete  examples  of 
"how  to  do"  rather  than  giving  concept  elaborations  dealing  with  "what  they  are"  can  reduce  initial  learning 
time.  In  addition,  increasing  elaborations  in  the  forms  of  graphical  aids,  textual  aids,  examples,  and 
explanations  is  effective  and  preferred  by  students  while  learning  a  computer-related  skill. 

However,  findings  on  the  role  of  text-based  elaborations  in  learning  a  procedure  are  inconsistent. 
Although  the  studies  previously  described  indicate  that  text-based  elaboration  facilitates  learning  a  procedure 
from  text,  other  studies  provide  evidence  that  unelaborated  texts  are  better  for  adult  learners  who  are  learning 
new  procedures.  For  instance,  some  studies  in  the  computer-related  domain  indicate  that  text-based 
elaborations  facilitate  learning  declarative  information  when  the  performance  measure  is  either  free  recall  or 
recall  of  the  main  points  of  the  text  (Reder,  Charney,  &  Morgan,  1986;  Pirolli  &  Anderson,  1985;  Pepper, 
1981;  Mayer,  1980).  In  contrast,  Carroll  (1984,  1990)  argues  that  self-learning  procedures  from  text  are 
more  efficient  when  the  text  contains  no  elaboration,  or  at  least  very  few  elaborations,  and  his  findings  from 
seveial  studies  have  indeed  shown  an  advantage  for  learning  procedures  from  unelaborated  text. 

Training 

During  the  1980s,  the  problem  of  training  users  in  the  use  of  computer  programs  has  been  the 
focus  of  considerable  research.  Carroll  and  colleagues  (1984,  1987-88)  at  IBM  pointed  out  several  problems 
that  novice  computer  users  had  while  learning  a  word  processing  program.  They  found  novice  learners  are 
apt  to: 

•  Be  overwhelmed  by  learning  tasks; 

•  "Jump  the  gun"  to  act  instead  of  reading  explicit  warnings  or  suggestions; 

•  Skip  the  information  that  cannot  be  executed; 

•  Trust  their  own  reason  to  act  instead  of  reading  the  instruction; 

•  Sometimes  ignore  the  screen  when  following  a  tutorial  step-by-step; 

•  Have  trouble  recovering  from  errors; 

•  Want  to  do  real  work  right  away  and  understand  "why"  and  "what"  they  are  doing. 

Minimalist  Approach.  Most  computer  learners  do  not  want  to  read  through  a  thick  manual  or 
work  through  an  on-line  tutorial  if  they  are  not  interested  primarily  in  the  to-be-learned  program  or  system. 
Impressed  by  the  frustration  and  failure  the  trainers  observed  when  novices  attempted  to  learn  to  use 
commercial  word  processing  systems  by  means  of  the  commercial  self-study  books  that  accompany  them, 
Carroll  et  al.  (1984)  developed  and  tested  the  Minimalist  Training  Model,  a  model  that  accommodates  and 
capitalizes  on  user  learning  styles  and  strategies.  This  is  what  Carroll  called  the  Minimalist  Approach 
which  attempts  to  support  active  learning  by  providing  concise  instruction  focused  on  easy-to-understand 
goals.  According  to  Carroll,  "the  key  idea  in  the  minimalist  approach  is  to  present  the  smallest  possible 
obstacle  to  the  learner's  efforts"  (Carroll,  1990,  p.77)  to  learn  and  acquire  competency. 

In  an  earlier  set  of  studies,  Carroll  et  al.  (1984,  1985)  created  the  "Minimal  Manual"  that 
incorporates  elements  of  the  Guided  Exploration  approach  (Carroll,  1984),  in  order  to  meet  the  needs  of 
computer  learners  in  learning  a  word  processing  system  with  "training  wheels."  By  training  wheels,  Carroll 
provided  new  users  with  a  real  but  easier  system  with  regard  to  learning  to  recognize  and  avoid  errors.  The 
training  wheels  subjects  were  better  able  to  transfer  their  skills  to  perform  advanced  full-system  editing 
functions  and  spent  less  time  in  error  recovery  than  full  system  subjects. 

Carroll  (1984,  p.  130)  outlined  five  premises  for  developing  computer  documentation  according  to 
the  Minimalist  model: 

•  "slash  the  verbiage"  by  eliminating  all  repetition,  summaries,  reviews,  practice  exercises,  the 
index,  and  the  troubleshooting  appendix; 

•  to  force  coordination  of  the  system  and  the  training: 

•  to  expect  every  possible  error: 

•  to  focus  on  real  tasks  and  activities:  and 

•  to  support  error  recognition  and  recovery.  ^  . 

<  'J  I 


680 


) 


After  a  series  of  studies,  Carroll  et  al.  (1984,  1985,  1987-88)  found  that  minimal  manuals  do  help  learners 
better  coordinate  their  attention  between  the  system  and  the  manual,  better  recognize  and  recover  from 
errors,  and  better  support  reference  use  after  training. 

Carroll  et  al.(1984,  1985)  advocated  a  Guided  Exploration  approach  to  provide  learners  with  brief 
training  materials  that  encourage  and  support  learning  by  exploration.  Guided  Exploration,  based  on  the 
principle  of  Minimal  Manual,  is  a  set  of  brief  cards  that  replace  a  commercial  self-instruction  manual.  In 
helping  people  to  use  a  word  processor,  Carroll  et  al.  found  that  brief  "guided  exploration"  cards  are  more 
effective  than  a  commercial  manual  in  many  respects.  Subjects  using  the  cards  accomplished  more  tasks, 
made  fewer  errors,  spent  less  time  reading  and  doing  the  tasks,  and  recovered  more  often  from  errors.  Brief 
guided  exploration  maicials  focus  users  on  the  steps  needed  to  accomplish  basic  tasks  by  encouraging  and 
supporting  exploration  through  easy-to-find  procedural  information  and  timely  error  recognition  and 
recovery  advice  (Catrambone  and  Carroll,  1987,  p.  170)  Recently,  Duin  (1990)  designed  a  study  that 
investigated  the  effects  of  a  minimal  manual  version  versus  a  guided  exploration  card  version  of 
documentation  for  learning  to  use  a  telecommunication  system.  The  results  showed  that  both  forms  of 
documentation—the  minimal  manual  and  the  guided  exploration  card—  are  viable  forms  for  the  design  and 
development  of  computer-related  documentation  because  they  both  center  on  the  learner  by  seeking  to  delete 
extraneous  text.  Duin  also  suggested  that  while  designing  documentation,  designers  should  (1)  center  on 
functions  rather  than  concepts,  (2)  think  in  terms  of  tasks  rather  than  terms,  and  (3)  develop  syntactic  rather 
than  conceptual  elaborations. 

Effective  computer  documentation  should  not  only  allow  learners  to  understand  how  to  perform 
specific  tasks,  but  should  also  allow  them  to  create,  explore,  and  integrate  knowledge  when  they  use  a 
computer  program  in  the  system.  While  computer  programs  as  a  whole  are  improving,  the  documentation 
that  accompanies  the  programs,  should  improve  along  with  it  to  better  meet  learners'  needs.  Toward  this 
end,  we  might  re-introduce  the  structure  of  a  manual  based  on  learners'  needs  by  codifying  some  of  the 
principles  which  were  helpful  to  the  guided  exploration  learners.  Summing  up  the  results  of  a  series  of 
studies,  Carroll  (1990)  concluded  and  developed  the  following  nine  principles  of  the  minimalist  approach 
(see  p.  78-93  for  a  review): 

1 .  Training  on  real  tasks-allow  the  user  select  meaningful  tasks  to  work  on. 

2.  Getting  started  fast-urge  getting  learners  started  doing  tasks  as  quickly  as  possible. 

3.  Reasoning  and  improvising—permit  self-directed  reasoning  and  improvising  in  learning 
process  to  enrich  their  own  training  experience. 

4.  Reading  in  any  order— eliminate  sequence  and  design  materials  allowed  learners  to  read  and 
use  in  any  order. 

5.  Coordination  system  and  training— impel  the  learners  to  attend  the  system  during  the  course 
of  training. 

6.  Supporting  error  recognition  and  recovery-support  the  recognition  of  and  recovery  from 
errors  so  that  errors  will  not  block  the  learners. 

7.  Exploiting  prior  knowledge— concrete  schemes  from  learner's  prior  knowledge  may  motivate 
and  facilitate  learning  and  skill  development. 

8.  Using  the  situation-let  the  situation  occur  naturally  rather  than  protect  the  learners  from 
experiencing  difficulties  or  errors. 

9.  Developing  optimal  training  designs— minimize  instructional  materials  that  obstruct 
learning  and  design  the  materials  to  support  learner-directed  activity  and  accomplishment. 

According  to  the  principle  of  the  minimalist  approach,  the  role  of  elaboration  that  Carroll 
emphasizes  is  learner-generated  elaboration  against  author-provided  elaboration.  He  encourages  learners  to 
engage  in  active  self-elaboration  (prompted  by  incomplete  materials  for  learning,  such  as  summaries  from 
the  full  text)  in  order  to  make  information  more  robust  and  accessible  in  memory  (Carroll  1990,  p.  81).  It 
appears  that  a  main  reason  the  performance  in  the  minimal  manual  is  superior  to  the  commercial  self- 
instruction  manual  is  that  it  forces  learners  to  make  inferences.  Actually,  Black,  Carroll,  and  McGuigan 
(1987)  found  that  the  Inferential  version  of  the  Minimal  Instruction  Manual  yielded  better  learning  than  did 
the  Skeletal,  the  Rehearsal  and  the  Lengthy  versions.  In  particular,  they  noted  that  the  Lengthy  Manual 
consistently  yielded  worse  performance  than  the  other  versions.  Thus,  continuing  research  is  investigating 
how  different  types  of  inference,  such  as  analogy,  proceduralization,  and  instantiation,  affect  learning. 

In  addition,  Carroll  (1984)  stated  that  "forget  the  preview  and  review;  studies  indicate  users  can 
learn  more,  given  less  information"  (p.  125).  He  strongly  recommends  eliminating  orientational  material, 
including  introduction,  overview,  preview,  review,  and  conceptual  elaborations  because  they  obstruct  the 
learner's  desire  to  do  something  instead  of  reading  about  how  to  do  something.  He  continues,  "the 
minimalist  approach  urges  getting  learners  started  doing  projects  as  quickly  as  possible,  allowing 
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meaningful  and  concrete  activity  to  provide  intrinsic  learning  giiidance-rather  than  relying  on  the  extrinsic 
guidance  of  conceptual  elaboration  and  practice  with  numbered  ste\s,  techniques  that  too  often  become 
obstacles  to  learning"  (Carroll,  1990,  p.81).  That  is,  the  Minimalist  design  presumes  that  if  you  give  the 
learner  less  (less  to  read,  less  overhead,  less  to  get  tangled  in),  the  learner  will  achieve  more. 

However,  a  truly  minimalist  approach— with  no  previews,  with  no  explanations  or  elaborations- 
may  not  be  best  in  all  situations.  For  example,  Reder,  Charney,  and  Morgan  ( 1986)  found  the  evidence  that 
performance  in  the  elaboration-after  condition  was  significantly  better  than  performance  in  the  unelaborated- 
after  condition,  where  no  source  of  elaborations  was  available.  But  in  the  same  study,  they  also  indicated 
that  when  subjects  already  knew  what  tasks  they  would  perform,  they  benefited  very  little  from  seeing 
examples  and  other  elaborations  in  the  text.  By  contrast,  Charney  and  Reder  (1987)  provided  the  value  of 
preview  that  learners  who  were  given  a  problem  first,  asked  to  try  to  solve  it,  and  shown  a  solution  at  the 
end,  learned  best. 

Therefore,  despite  its  brevity,  Wendt  (1991)  argued  that  the  Minimal  Manual  (Carroll,  1990) 
should  not  be  thought  of  as  simply  an  unelaborated  version  of  the  original  text  for  several  reasons:  (1) 
information  was  eliminated  that  was  not  task-oriented(  e.g.,  descriptions  of  the  system);  (2)  information 
was  added  to  address  error  recognition  and  recovery  procedures;  (3)  information  was  rearranged  to  support 
topics  of  interest  to  learners;  and  (4)  sometimes  procedural  details  were  deliberately  specified  incompletely 
or  introduced  with  an  invitation  to  try  them  and  see  what  would  happen. 

In  conclusion,  the  effects  of  minimalist  approach  cannot  separate  from  the  effects  of  the  guided 
exploration  (discovery)  approach  as  well  as  the  emphasis  on  error  recognition  and  recovery.  Nevertheless, 
using  a  minimalist  approach,  people  learning  a  procedure  from  text  are  able  to  acquire  competence  and 
perforin  well  on  transfer  tasks  without  benefit  of  many  author-provided  elaborations.  Besides  author- 
provided  elaborations,  the  minimalist  approach  allows  learners  to  generate  their  own  elaborations  during  the 
learning  process.  That  is,  author-provided  elaborations  may  facilitate  skill  learning  in  some  situations,  but 
not  others.  Therefore,  for  developing  an  effective  text,  it  is  very  important  to  establish  guidelines  for 
determining  what  types  of  elaborations  are  optimal  in  what  situations  and  when  to  use  them  in  order  to 
facilitate  fact  or  skill  learning. 

Minimalist  vs  Maximalist  Approach.  To  answer  the  question  whether  computer  manuals 
should  be  minimalist  and  contain  only  information  that  is  absolutely  necessary,  or  should  be  maximalist 
and  fully  explain  unfamiliar  concepts  and  procedures,  Wendt  ( 1991 )  directly  compared  three  versions  of  texts 
in  order  to  examine  the  effects  of  an  original  version,  as  well  as  an  Elaborated  and  an  Unelaborated  revisions 
of  an  excerpt  from  a  commercial  system  user's  manual  on  the  self-learning  procedures  from  text.  First,  she 
summarized  the  issues  that  support  an  extensively  elaborated  text  and  those  that  support  a  minimally 
elaborated  self-instructional  text,  and  then  provided  contrasting  guidelines  for  developing  an  effective  text 
(Wendt  1991,  p.36). 

The  guidelines  for  supporting  an  extensively  elaborated  text  to  produce  an  effective,  readable  text 
for  the  learners  (at  least  in  the  short  run)  are  the  following: 

•  Use  concrete  examples  to  support  every  major  and  most  minor  points. 

•  Use  graphical  aids  such  as  illustrations  and  diagrams. 

•  Use  textual  a.ds  such  as  introductions,  summaries,  and  glossaries. 

•  Use  explanations  for  all  definitions,  technical  terms,  solution  rules,  and  procedures. 

•  Use  syntactic  elaboration  instead  of  conceptual  elaboration  to  clarify  how  to  perform  the  to-be- 
learned  procedures. 

Guidelines  for  supporting  a  minimally  elaborated  text  to  help  learners  get  started  faster,  perform 
better,  make  fewer  errors,  and  recover  from  errors  more  successfully,  are  the  following: 

•  Do  not  embed  examples  which  may  be  obstructive  for  the  learners. 

•  Do  not  include  explanations  but  urge  the  learners  to  discover  their  own. 

•  Do  not  provide  materials  that   ive  the  learners  a  model  of  correctness;  but  encourage 
exploration  and  support  error  i  covery. 

•  Do  not  increase  text  by  adding  elaboration  but  reduce  text  by  eliminating  elaboration  and 
instruction  for  the  purpose  of  requiring  the  learners  to  make  inferences  from  the  incompletely 
remaining  text. 

Wendt  (1991)  offered  a  new  perspective  for  evaluating  the  role  of  text-based  elaboration  in  the 
process  of  self-learning  a  new  procedure  by  directly  comparing  the  effects  of  these  two  approaches  on 
learning  the  same  procedure  and  performing  the  same  tasks  in  one  study. 

According  to  the  guidelines  mentioned  above,  the  Elaborated  version  in  Wendt's  study  included 
changes,  such  as  (1)  reorganizing  the  content  by  separating  conceptual  and  procedural  information;  (2) 


adding  elaboration  in  the  form  of  examples,  restatements,  explanations,  an  introduction,  a  summary;  (3) 
adding  text-to-graphic  references  to  support  both  conceptual  information  and  procedural  information,  (4) 
adding  several  examples  of  using  different  combinations  of  connectors  to  link  selection  rules;  and  (5) 
removing  side-margin  subheadings.  By  following  the  principles  of  designing  minimalist  instructions, 
changes  in  the  Unelaborated  version  included  (1)  identifying  statements  of  procedures  and  rules;  (2)  deleting 
all  elaboration  such  as  examples,  restatements,  and  descriptions  of  what  the  software  does  as  a  result  of 
procedures;  (3)  deleting  unnecessary  words;  (4)  deleting  introductory  text;  and  (5)  deleting  side-margin 
subheadings  (Wendt,  1991). 


Changes  from  the  Original  User's  Manual    in  Wendt's  Study 


Elaborated  Text  version 

( 1 )  reorganizing  the  content  by  separating 

conceptual  and  procedural  information 

(2)  adding  elaboration-- 

•  examples 

•  restatements 

•  explanations 

•  an  introduction 

•  a  summary 

(3)  adding  text-to-graphic  references  to  support 

both  conceptual  information  and  procedural 
information 

(4)  adding  several  examples  of  using  different 

combinations  of  connectors  to  link  selection 
rules 

(5)  removing  side-margin  subheadings 


Unelaborated  Text  version 

(1)  identifying  statements  of  procedures  and 

rules 

(2)  deleting  all  elaboration- - 

•  examples 

•  restatements  (actually  included  three) 

•  explanations 

•  descriptions  of  what  a  procedure  goes 

•  an  introduction 

•  a  summary 

(3)  deleting  unnecessary  words 

(4)  deleting  introductory  text 

(5)  deleting  side-margin  subheadings 


Consistent  with  the  idea  that  text-based  elaborations  facilitate  learning  declarative  information 
when  the  performance  measure  is  either  free  recall  or  recall  of  the  main  points  of  the  text,  Wendt  indicated 
in  her  study  that  the  performance  of  the  Elaborated  text  on  the  given  tasks  was  equal  or  superior  to  the 
Original  text  and  the  Unelaborated  text  versions.  She  found  that  those  who  read  the  Elaborated  text  achieved 
a  higher  percentage  of  elements  correct,  more  correct  solutions,  committed  fewer  errors,  did  not  spend 
significantly  more  time  reading  their  text  but  spent  less  time  working  on  the  application  tasks,  and  rated 
their  text  more  positively  and  more  consistently.  She  also  indicated  that  the  subjects  who  studied  the 
Elaborated  text  continued  to  perform  well  as  task  difficulty  increased  because  the  Elaborated  text  in  her 
design  separated  conceptual  and  procedural  information  and  elaborated  both  types  of  information  that  served 
an  important  factor  in  facilitating  their  understanding.  However,  she  reported  that  not  all  subjects  who  read 
the  Elaborated  text  performed  well.  In  other  words,  not  all  people  need  elaboration  to  develop  a  good 
understanding  in  a  procedure  learning. 

While  Wendt  wanted  her  subjects  to  read  the  instructional  text  first  and  then  complete  the 
application  tasks  'without'  the  text,  however,  it  belongs  to  an  immediate  posttest  to  test  subjects'  memory 
for  information  acquired  via  text  after  study.  It  occurred  to  me,  that  in  a  real  world  setting,  learners  may  not 
need  recall  text  from  their  memory  in  order  to  learn  a  new  computer  program.  On  the  one  hand,  Carroll 
(1984)  argued  that  if  learners  go  along  nose-in-the-book,  they  will  not  learn  to  use  the  system  (p.  126).  On 
the  other  hand,  focusing  on  the  learner's  primary  goal  to  complete  a  specific  task,  learners  want  to  attempt 
tasks  immediately  rather  than  read  and  remember  information  for  future  recall.  Therefore,  the  idea  of 
coordination  of  the  system  and  the  training  at  the  same  time  should  be  concerned  for  further  studies. 

The  Current  Study 

The  purpose  of  study  in  this  paper  is  to  investigate  the  effects  of  augmenting  and  deleting 
elaborations  in  an  existing  self-instructional  text  from  a  computer  automated  database  file.  The  intent  here 
is  to  replicate  and  extend  Wendt's  (1991)  study  by  asking  students  to  read  one  of  the  three  self-instructional 
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texts  and  complete  several  given  tasks  accompanied  with  the  copy  of  materials  in  the  same  period  of  time 
instead  of  an  immediate  posttest.  To  increase  the  power  of  the  analysis,  the  present  r.tudy  increases  the 
sample  size  from  21  to  60.  The  study  attempts  to  discover  whether  students  who  learned  to  use  a  computer 
database  program  from  an  elaborated  text  did  better  than  students  who  learned  from  an  unelaborated  one. 

Learner-generated  elaborations  were  not  included  in  the  texts,  since  it  is  too  difficult  to  control  for 
differences  in  learner's  prior  knowledge.  The  present  study  will  exclude  all  types  of  learner-generated 
elaborations  in  favor  of  author-provided  elaborations.  The  types  of  elaborations  provided  by  the  experienced 
author  in  the  study  are  those  directly  included  in  the  material  to-be-learned  for  the  purpose  of  facilitating  a 
more  complete  and  accurate  understanding  of  the  material  itself,  and  appropriate  application  to  individual 
tasks.  Specially,  the  research  questions  asked  here  are:  (I)  will  author-provided  elaborations  facilitate  the 
learners'  performance  on  given  tasks  in  a  self-learning  procedure  from  text?  (2)  will  the  elaborated  text  be 
more  effective  than  the  unelaborated  text  during  the  self-learning  procedure?  (3)  will  the  amount  of  text- 
based  elaboration  differentially  affect  performance  on  the  given  tasks  while  the  difficulty  of  the  tasks 
increases? 

Method 

The  present  study  sought  to  extend  the  previous  investigation  by  Wendt  (1991 )  to  examine 
whether  students  who  learn  from  an  elaborated  text  do  better  on  performance  of  a  computer  database 
application  than  students  who  learn  from  an  unelaborated  text.  A  purpose  of  this  study  was  to  replicate 
previous  findings  reported  by  Wendt  ( 1991),  so  the  materials  used  here  are  similar,  with  the  exception  of  the 
number  of  the  subjects  and  the  way  of  the  procedures. 

Subjects 

Sixty  undergraduate  students  from  the  Department  of  Psychology  at  the  University  of  Minnesota 
completed  the  study.  Students  received  research  participation  points  as  well  as  five  dollars  for  participating. 
The  subjects  varied  widely  in  their  prior  general  experience  with  computers;  however,  they  had  no  prior 
experience  with  the  Microsoft  Works  Database  application. 

Materials 

The  database.  Subjects  used  a  database  file  of  25  records  created  in  the  Microsoft  Works 
Database  program  in  conjunction  with  the  six  application  tasks  Although  all  25  records  contained  the 
same  1 5  fields,  data  in  each  field  were  different  from  one  record  to  another.  The  contents  of  the  1 5  fields 
within  each  record,  including  name,  title,  company,  and  desired  sessions,  consisted  of  fictitious  information 
from  a  registration  form  requested  for  a  training  workshop. 

The  Original  text.  The  text  was  adapted  from  the  Microsoft  Works  User's  Manual  (1989).  an 
instructional  text  provided  for  the  learners  to  use  and  apply  the  computer  software— Microsoft  Works.  In 
this  study,  the  six  pages  that  explain  how  to  use  the  Selection  Rules  to  locate  and  retrieve  desired 
information  from  the  database  file  were  excerpted  from  the  User's  Manual.  Two  versions  were  created  from 
the  Original  Text  by  Wendt  (1991)  according  to  the  guidelines  she  set  (discussed  in  the  previous  section). 
One  was  an  Elaborated  Text  version  that  was  developed  using  the  guidelines  for  an  extensive  Elaboration 
approach.  The  other  was  an  Unelaborated  Text  version  developed  using  the  guidelines  for  designing 
minimalist  instructions. 

Elaborated  text  version.  The  Elaborated  Text  version  was  extended  to  twenty-two  pages 
long.  According  to  the  guidelines  established  by  Wendt  (1991),  the  Elaborated  Text  version  was  developed 
from  the  Original  Text  in  three  steps: 

First,  the  content  from  the  Original  Text  was  analyzed  and  outlined.  There  were  four  major  areas 
identified:  ( 1 )  deciding  selection  rules;  (2)  entering  selection  rules;  (3)  connecting  selection  rules;  and  (4) 
making  changes. 

Second,  a  terminal  performance  objective  was  identified  and  a  learning  hierarchy  was  constructed. 
Since  the  terminal  performance  objective  in  the  learning  procedure  was  to  retrieve  records  from  a  database 
file  by  using  selection  rules,  a  learning  hierarchy  that  s!iowed  the  prerequisite  relationships  among  the  skills 
to-be-learned  played  an  important  role.  As  a  result,  the  Elaborated  Text  version  was  constructed  with  the 
following  components:  (1)  an  introduction;  (2)  how  to  create  a  single  selection  rule;  (3)  how  to  enter  a 
single  selection  rule;  (4)  how  to  use  combinations  of  selection  rules;  (5)  how  to  enter  multiple  selection 
rules,  (6)  how  to  make  changes;  and  (7)  a  summary. 

Third,  syntactic  elaborations  (rather  than  conceptual  elaborations)  were  created  and  extensive 
elaborations  were  provided  to  support  the  explanations  of  procedures  (such  as  how  to  create  a  single 
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selection  rule  and  how  to  link  selection  rules  using  connectors).  The  most  frequently  used  type  of 
elaboration  was  examples  (e.g.,  create  the  pairs  of  correct  and  incorrect  examples  in  terms  of  supplementing 
the  explanation  of  why  the  incorrect  examples  were  not  right). 

There  were  some  elaborations  that  provided  additional  content  which  dealt  with  how  to  use  the 
connectors  in  these  ways.  To  begin  with,  some  examples  were  given  to  explain  the  use  of  connector 
"And."  Then,  some  examples  were  given  to  explain  the  use  of  connector  "Or."  Finally,  several  examples 
using  a  combination  of  the  two  connectors  were  given  and  explained. 

Unelaboratcd  text  version.  The  Unelaborated  Text  version  was  six  pages  long.  Working 
from  the  Original  Text,  this  version  was  created  according  to  the  principles  of  designing  minimalist 
instructions:  (1 )  identify  statements  of  procedures  and  rules;  (2)delete  all  elaborations  including  examples, 
restatements,  explanations,  and  descriptions  of  what  a  procedure  goes;  (3)  delete  introductory  text;  (4)  delete 
unnecessary  words;  and  (5)  delete  side  margin  subheadings. 

As  a  result,  114  lines  of  text  in  the  Original  Text  were  reduced  to  75  lines  of  text.  However,  the 
format  of  the  original  text  was  maintained  in  the  Unelaborated  Text  version.  After  a  pilot  study,  the 
Unelaborated  Text  was  revised  by  adding  three  restatements  (1)  to  explain  the  definition  of  "record 
comparison  information";  (2)  a  step  in  the  procedure  for  entering  a  selection  rule  on  the  computer  screen; 
and  (3)  how  the  connectors  interacted  when  they  were  used  together.  In  spite  of  these  additions,  it  was 
consistent  with  the  principle  of  minimalist  approach  advocated  by  Carroll  (1990),  including  error 
recognition  and  recovery  procedures. 

A  comparison  of  frequencies  of  different  types  of  elaboration  across  the  three  different  types  of 
texts  is  listed  in  the  following  table.  Three  illustrations  that  provided  important  navigation  information 
were  included  in  all  three  versions  . 

Comparison  of  Frequencies  of  Different  Types  of  Elaboration 


Original 

Elaborated 

Unelaborated 

Types  of  Elaborations 

Text 

Text 

Tex.t 

(6  pages) 

(22  pages) 

(6  pages) 

Boldface  Types 

2  items 

all  items 

all  items 

Italics 

yes 

yes,  with  some 

yes, 

single  and  double 

with  only  1  single 

underline 

underline 

Graphical  Illustrations 

3 

3 

3 

Textual  Examples 

6 

31 

0 

Marginal  Comments 

6 

0 

0 

Paraphrases  and  Restatements 

0 

10 

3 

Headings 

4 

7 

5 

Summaries 

0 

1 

0 

Overviews 

1 

7 

5 

Intertextual  References 

1 

1 

1 

Text-to-Graphic  References 

0 

L, 

18 

3 

Background  information  and  prior  experience  questionnaire.  This  questionnaire 
contained  six  questions  asking  the  participants  to  provide  (I )  academic  major,  (2)  year  in  school,  (3)  status 
(full  or  part-time  student),  (4)  prior  experience  with  computer  hardware,(5)  prior  experience  with  computer 
software  fcr  what  purpose,  and  (6)  prior  experience  with  the  Microsoft  Works  database  application. 

Background  information  about  these  materials  preface.  A  three-page  supplement  to 
the  text  was  created  to  demonstrate  what  a  database  is,  how  it  works,  and  to  explain  the  contents  of  the 
database  file  that  would  be  used  in  the  study.  The  same  three-page  supplement  was  placed  directly  above 
each  text  and  marked  as  pages  1,  2  and  3.  Initially,  page  1  established  a  context  both  for  the  reading  of  text 
and  the  given  application  tasks  in  order  to  provide  a  general  overview  of  this  study.  Page  2  explained  how 
information  is  organized  in  a  database  file,  and  provided  an  example  for  subjects  to  distinguish  two  key 
terms:  records  and  fields.  Page  3  listed  the  15  fields  with  their  definitions.  Therefore,  the  text  that  described 
the  use  of  selection  rules  always  started  on  page  4. 
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Application  tasks.  There  were  six  application  tasks  designed  to  test  subjects'  understanding  of 
the  information,  procedures  provided  in  the  text,  and  their  ability  to  apply  the  information  in  the  computer 
setting.  For  example,  task  1  asked:  "Who  is  coming  from  IBM?"  which  required  subjects  to  retrieve  the 
records  of  the  persons  who  were  coming  from  IBM.  Since  IBM  belongs  to  a  "company"  (its  corresponding 
field),  subjects  should  have  selected  "company"  from  the  field  first,  chosen  "equals  to"  from  the  comparison 
phrases,  and  then  typed  "IBM"  in  the  box  of  record  comparison  information.  This  task  was  the  simplest 
one,  which  used  only  one  selection  rule  without  any  logical  connectors.  Task  2  asked:  "Which  companies 
are  sending  either  the  President  or  the  CEO?"  which  required  retrieving  the  records  of  companies  that  were 
sending  the  persons  whose  title  was  either  President  or  CEO.  Since  the  box  for  record  comparison 
information  can  only  hold  one  instruction  at  a  time,  subjects  were  required  to  use  a  logical  connector  "Or" 
to  link  the  person  whose  "title"  "equals  to"  "President"  and  whose  "title"  "equals  to"  "CEO".  Therefore, 
two  selection  rules  were  used  in  this  task. 

The  difficulty  level  of  the  tasks  on  the  list  increased  by  (1)  increasing  the  number  of  selection  rules 
required  to  complete  a  task;  and  (2)  increasing  the  correct  use  of  logical  connectors  to  combine  the  selection 
rules.  The  following  table  (adapted  from  Wendt  1991,  p.48)  shows  the  increase  in  difficulty  by  task.  To 
clarify,  when  rules  are  linked  with  a  combination  of 'And'  and  'Or',  the  program  will  locate  only  those 
records  that  satisfy  all  the  Selection  Rules  continuously  linked  by  'And'.  When  it  sees  the  Connector  'Or', 
it  will  begin  a  new  and  separate  search  for  these  records. 

Dimensions  of  Difficulty  in  the  Application  Tasks 


Task  Number  Selection  Rules  Logical  Connectors 

1  1  0 

2  2  1  -  Or 

3  3  2  -  And,  And 

4  4  3  -  And,  Or,  And 

5  5  4  -  And,  Or,  And,  And 

6  6  5  -  And,  And,  And,  And,  And 


Attitude  survey.  The  attitude  survey  was  composed  of  six  questions.  Responses  were  recorded 
by  subjects  using  a  six-point  scale.  Four  of  the  questions  used  by  Pepper  (1981)  were  adapted  in  order  to 
compare  the  results.  These  four  questions  asked  the  subjects  (1)  how  easy  the  text  was  to  understand,  (2) 
how  well  the  material  was  "taught"  by  the  text,  (3)  how  well  reader's  questions  were  answered,  and  (4)  asked 
for  an  "overall"  rating.  In  addition,  two  questions  were  added  to  obtain  feedback  on  the  examples  and  text- 
to-graphic  illustrations,  since  many  of  them  were  added  to  the  Elaborated  Text  version. 

Design 

This  study  involved  a  single  independent  variable  with  three  levels:  Original  Text  version. 
Elaborated  Text  version  and  Unelaborated  Text  version.  The  dependent  measures  within  the  study  were  ( 1 ) 
percentage  scores  of  elements  correct  on  each  of  six  application  tasks,  (2)  number  of  correct  solutions  across 
all  application  tasks,  (3)  total  time  spent  on  reading  the  text  and  working  with  the  application  tasks,  and  (4) 
results  on  an  attitude  survey.  The  results  of  each  of  the  six  performance  tasks  were  analyzed  in  a  repeated 
measures  fashion  with  one  between-subjects  factor  ("Version"  with  three  levels)  and  one  within-subjects 
factor  ("Task"  with  six  levels).  One-way  ANOVAs  were  used  to  analyze  the  results  for  each  of  the  other 
measures,  with  Version  as  the  independent  variable.  Results  of  mean  scores  across  the  three  versions  from 
attitude  survey  were  also  analyzed  with  one-way  ANOVA  and  presented  in  simple  descriptive  format. 

Procedures 

The  study  was  conducted  individually  in  one  session  approximately  60  -  90  minutes  in  length. 
Sixty  copies  of  materials,  including  20  of  the  Original  Text,  20  of  the  Elaborated  Text,  and  20  of  the 
Unelaborated  Text  vers  ions,  were  prepared  prior  to  the  study  and  placed  in  unmarked,  brown  envelopes.  To 
achieve  random  distribution,  all  envelopes  were  repeatedly  shuffled  so  that  subjects  studied  one  text  in  each 
of  the  three  conditions:  Original,  Elaborated  or  Unelaborated  Text  version. 

After  completing  a  Consent  Form,  a  Background  Information  Questionnaire,  and  an  Experience 
Questionnaire,  each  participant  was  asked  to  read  the  first  three  pages  of  materials  (that  provided  information 
about  what  a  database  is  and  how  to  perform  the  procedures  in  the  study).  No  time  limitation  was  imposed 
as  subjects  familiarized  themselves  with  the  content  of  these  three  pages.  When  a  subjects  was  ready,  a  list 
of  tasks  was  given  and  he/she  was  directed  to  read  either  an  Original,  an  Elaborated  or  an  Unelaborated 
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instructional  text  of  Microsoft  Works  Database  application,  and  to  complete  the  six  given  tasks  within  an 
hour.  The  instructional  texts  were  available  at  the  same  time  that  the  subjects  were  completing  the  tasks. 
Subjects  were  allowed  to  mark  or  underline  the  text  while  reading  if  they  so  desired,  but  they  could  not  ask 
any  questions.  When  the  hour  was  up,  subjects  were  told  to  stop  working,  whether  they  completed  all  the 
application  tasks-or  not.  Subjects  could  stop  at  any  time  during  the  session  if  they  finished  the  tasks 
earlier.  Then,  the  six-item  attitude  survey  was  given.  Participants  were  asked  to  rate  each  question  with 
"1"  being  the  most  negative  reaction  and  "6"  being  the  most  positive  reaction.  Finally,  each  subject 
received  research  participation  points,  as  well  as  five  dollars,  for  participating,.  Subjects  were  also  given  a 
copy  of  the  Education  Debriefing  Form  that  briefly  summarized  the  rationale  of  this  research  on  the  role  of 
elaboration,  and  provided  references  and  additional  sources  of  information  on  this  research  topic. 


Results 

Mean  Percentage  of  Elements  Correct  on  Each  Application  Task 

Table  1  presents  the  mean  percentage  of  elements  correct  for  each  text  version  and  application  task. 
Results  of  a  repeated  measures  ANOVA  using  Versions  (Original,  Elaborated,  and  Unelaborated)  as  the 
between-subjects  factor  and  Task  (1  to  6)  as  the  within-subjects  factor  was  shown  in  Table  2  as  follows:  the 
between-subjects  factor  of  Version  was  not  significant  [F(2,  57)  =  .36,  p  =  .701,  MSe  =  2597.281;  the 
within-subjects  factor  of  Task  was  significant  [F(5,  285)  =  22.22,  p  <  .001,  MSe  =  421.05],  while  the 
Version  by  Task  interaction  effect  was  not  [F(10,  285)  =  .90,  p  =  .530  ,  MSe  =  421.05]. 


Table  1 :  Mean  Percent  of  Elements  Correct  for  Each  Version  and  Task 


Text  Version 

Task  I 

Task  2 

Task  3 

Task  4 

Task  5 

Task  6 

Task  Mean 

Original 
(n=20) 

100% 

91.45% 

85.45% 

66.60% 

70.80% 

60.75% 

78.800% 

Elaborated 
(n=20) 

100% 

91.45% 

77.30% 

72.95% 

82.45% 

72.10% 

82.708% 

Unelaborated 
(n=20) 

95.05% 

87.15% 

73.60% 

63.30% 

77.15% 

67.10% 

78.342% 

Total  Mean 
(n=60) 

98.35% 

90.02% 

78.78% 

67.62% 

76.80% 

66.65% 

79.703% 

Table  2:  Analvsi 

s  of  Variance  for  Percentage  of  Elements 

Correct 

Source 


Sum  of  Squares       Degrees  of        Mean  Square 
Freedom 


Between  Subjects 
Version 

Subjects  Within  Groups 


1854.16 
148044.88 


57 


927.08 
2597.28 


.36 


.701 


Within  Subjects 
Task 

Version  x  Task 

Task  x  Subjects  Within 

Groups  


46788.95 
3806.74 
120000.48 


5 

10 

285 


9357.79 
380.67 
421.05 


22.22 
.90 


.000 
.530 
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A  follow-up  series  of  t-tests  was  run  pairwise  across  the  six  tasks.  A  Bonfcrroni  procedure  was 
used  to  control  Type  I  error  resulting  in  alpha  =  .01.  Results  of  the  follow-up  t-lests  to  the  "Task"  repeated 
measures  effect  are  shown  in  Table  3.  To  summarize  Table  3,  there  were  significant  difference  between 
Task  1  and  Tasks  2,  3,  4,  5,  and  6;  similarly,  Task  2  was  different  from  Tasks  3,  4,  5,  and  6.  Moreover, 
Task  4  was  significantly  different  from  Tasks  3  and  5. 


Table  3:  Pairwise  t-tests  (df  =  59)  Amonp  the  Six  Tasks 
(  alpha  =  .01,  using  Bonferroni  adjustment  ) 


liisk 


lask 

4 


t  =  4.1 1 
p  =  .000* 


t  =  5.68 
p  =  .(XX)* 

t  =  3.28 
p  =  .002* 


8.97 

t  =  5.80 

t  =  6.63 

.(XX)* 

p  =  .000* 

p  =  .000* 

6.88 

t  =  4.06 

t  =  5.71 

.000* 

p  =  .000* 

p  =  .000* 

3.29 

t  =  .54 

t  =  2.44 

.002* 

p  =  .594 

p  =  .018 

t  =  -2.85 

t  =  .22 

p  =  .006* 

p  =  .824 

t  =  2.47 

p  =  .0 1 6 

Number  of  Correct  Solutions 

"Number  of  Correct  Solutions"  means  the  number  of  subjects  who  achieved  1 00%  elements  correct 
for  each  task  in  each  text  version  group.  Table  4  shows  the  number  of  correct  solutions  for  each  version 
and  task.  However,  there  was  no  difference  between  tasks  in  terms  of  the  total  number  of  correct  solutions 
|F(2,57)  =  .573,  p  =  .567,  MSe  =  3.227). 

Table  4:  Number  of  Correct  Solutions  for  Each  Version  and  Task 


Text  Version 


Task  I        Task  2        Task  3        Task  4       Task  5       Task  6 


Total 


Original 
(n=20) 


20 


14 


70 


Elaborated 
(n=20) 


20 


14 


13 


71 


Unelaborated 
fn=20) 


17 


12 


60 


a 
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Total  Time  Spent  Reading  and  Working  on  Tasks 

Table  5  presents  the  mean  time  spent  for  each  text  version  group.  The  result  shows  no  differences 
for  total  time  on  reading  and  working  the  tasks  among  the  three  Text  Versions  [F  =  .175,  p  =  .840,  MSe  = 
125.507]. 

Table  5:  Mean  Time  Spent  Reading  and  Working  on  Tasks 


Text  Version 

Time 

Original 

46.15  min. 

Elaborated 

48.15  min. 

Unelaborated 

46.60  min. 

Attitude  Survey 

Table  6  presents  the  results  of  the  attitude  survey  ("1"  was  the  least  favorable  reaction  and  "6"  was 
the  most  positive  reaction).  The  data  seems  to  indicate  that  not  all  subjects  needed  examples  or  illustrations 
for  help  among  the  three  conditions.  Although  on  average  subjects  in  all  three  groups  rated  the  texts  in  a 
positive  way,  the  ANOVA  on  mean  attitude  score  across  the  three  versions  was  non-significant  [F(2,  57)  = 
.224,  p  =  .800,  MSe  =  1 .024].  No  differences  are  on  each  question  among  the  three  versions,  either. 

Table  6:  Ratings  of  the  Three  Text  Versions 


Questions  of  Attitude 

Original 

Elaborated 

Unelaborated 

Survev 

Mean 

Range 

Mean 

Mean 

Range 

1 .  Easy  to  Understand 

3.65 

1-6 

3.85 

2-6 

3.55 

1-5 

2.  Taught  Material  Well 

4.10 

2-6 

4.10 

2-6 

3.85 

1-6 

3.  Answered  Questions 

3.65 

1-6 

3.95 

2-6 

3.85 

1-6 

4.  Illustrations  Helpful 

4.05 

1-6 

4.15 

2-6 

4.35 

3-6 

5.  Examples  Helpful 

4.05 

1-6 

4.20 

2-6 

3.75 

2-6 

6.  Overall 

3.90 

2-6 

4.15 

2-6 

3.85 

1-6 

Group  Mean  of  Attitude 

3.90 

4.07 

3.87 

Discussion 

The  present  study  examined  the  effect  of  varying  degrees  of  text  elaborations  on  self-directed 
student  learning,  to  learn  to  use  a  computer  database  application.  Using  guidelines  from  Wendt's  (1991 ) 
study,  an  Elaborated  Text  and  an  Unelaborated  Text  were  developed  from  an  original  commercial  system 
user's  manual.  Subjects  completed  six  applied  tasks  by  reading  one  of  the  three  instructional  text  versions, 
and  completed  an  attitude  survey  at  the  end. 

The  results  of  the  current  study  not  only  deviate  from  Wendt's  findings  (which  supported  the  effects 
of  elaboration  on  self-directed  learning  from  text),  but  also  differ  from  Carroll's  approach,  which  suggests 
the  minimalist  manual's  superiority  to  the  commercial  self-instruction  manual.  The  effects  of  task  in  a 
3(Version)  x  6(Task)  repeated-measures  ANOVA  was  significant,  while  both  the  Version  and  the  Version  x 
Task  interaction  were  not.  No  differences  among  conditions  were  found  for  treatment  with  the  following 
dependent  variables:  (a)  percentage  of  elements  correct  on  each  of  six  performance  tasks,  (b)  number  of 
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correct  solutions  across  all  performance  tasks,  (c)  total  time  spent  reading  and  working  on  the  application 
tasks,  and  (d)  results  on  attitude  survey.  It  seems  that  the  advantages  of  elaborations  on  computer 
documentation  do  not  produce  the  same  effect  when  subjects  can  complete  the  application  tasks 
accompanied  by  the  instructional  text.  That  is,  unlike  Wendt's  results,  subjects  who  read  manuals 
containing  extensive  elaborations  in  the  present  study  did  not  out-perform  subjects  who  read  manuals  with 
less  elaboration.  And,  unlike  Caroll's  findings  (1984,  1985,  1987-88,  1990),  subjects  who  read  the 
unelaborated  manual  did  not  perform  better  than  subjects  who  read  the  longer,  elaborated  manual. 

A  possible  reason  for  thr  differences  in  results  reported  by  Wendt  (1991)  is  that  the  present  study 
involved  simultaneously  completing  the  tasks  and  reading  the  text  in  order  to  simulate  goal-oriented 
learning  in  a  real-world  setting.  Subjects  in  the  current  study  attempted  the  application  tasks  accompanied 
by  the  text  instead  of  reading  the  text  first  and  then  working  on  ihe  tasks  without  using  reading  materials. 
This  was  done  to  maximize  the  external  validity  of  the  study,  because  few  people  learn  to  navigate  a  new 
computer  program  by  memori/.ing  an  instructional  text.  In  Wendt's  study,  results  focused  on  the 
effectiveness  of  elaborations  on  recall  and  retrieval  of  the  main  points  of  the  text.  In  the  current  study,  the 
investigation  focused  on  whether  text-based  elaborations  facilitate  procedural  learning  while  learning  a  new 
computer  program  in  a  real-world  setting. 

An  analysis  of  subjects'  performance  on  the  easier  versus  the  more  difficult  tasks  gives  additional 
information.  The  six  application  tasks  were  created  based  on  a  learning  hierarchy  for  the  procedural 
learning,  and  most  subjects  correctly  completed  the  easiest  task,  which  required  the  use  of  only  one  single 
selection  rule.  As  task  difficulty  increased,  subjects  were  required  to  construct  and  input  more  rules,  as  well 
as  combine  those  rules  by  using  the  correct  connectors. 

Results  show  that  each  of  the  six  tasks  seemed  successively  more  difficult,  but  Task  4  appeared  the 
most  difficult.  The  reason  for  this  might  be  not  attributed  to  subjects'  understanding  of  the  logical  concepts 
of  the  connecters  And'  and  'Or',  rather  than  how  to  select  and  input  the  rules.  Task  4  asked  "Who  is 
coming  from  Texas  and  California  that  is  attending  the  Session  on  Topic  C?"  It  required  retrieving  the 
records  of  tho  persons  who  come  from  the  states  of  either  the  Texas  or  California,  but  they  should  attending 
Topic  C.  Many  subjects  used  the  following  rules: 

the  persons  whose  "state"  "equals  to"  "Texas" 

'Or'  "state"  "equals  to"  "California", 

And'  "attending  Topic  C"  "equals  to"  "yes". 

Using  these  rules,  the  answer  obtained  shows  the  persons  who  come  from  Texas  but  not  attending  the 
session  on  Topic  C,  because  the  connector  'And'  only  linked  with  the  last  two  rules:  "state  equals  to 
California"  and  "attending  Topic  C".  The  correct  rules  are  the  following: 

the  persons  whose  "state"  "equals  to"  "Texas" 

'And'  "attending  Topic  C"  "equals  to"  "yes". 

'Or'  "state"  "equals  to"  "California". 

And'  "attending  Topic  C"  "equals  to"  "yes". 

Four  selection  rules  with  three  connectors  should  have  been  used  in  this  task. 

There  were  six  examples  given  in  the  Elaborated  text,  three  examples  in  the  Original  Text,  and 
none  in  the  Unelaborated  text.  The  investigator  expected  that  the  subjects  using  the  Elaborated  text  would 
perform  better  because  more  examples  were  provided  that  supported  the  solution.  As  a  result,  subjects  who 
read  the  Elaborated  Text  version  scored  72.95%  of  elements  correct  in  Task  4,  compared  with  a  score  of 
66.60%  of  elements  correct  for  subjects  who  read  the  Original  Text  version  and  63.30%  of  elements  correct 
for  subjects  who  read  the  Unelaborated  Text  version.  However,  these  differences  were  not  statistically 
significant.  When  looking  at  the  number  of  correct  solutions  on  the  same  task,  only  6  out  of  20  subjects 
got  every  item  correct  in  the  Elaborated  Text  group,  only  5  of  them  got  every  item  correct  in  the  Original 
Text  group,  and  4  of  them  got  every  item  correct  in  the  Unelaborated  group.  The  performance  on  Task  4 
was  poor  across  all  three  conditions.  This  finding  did  not  support  the  view  that  using  examples  is  an 
important  tool  for  bridging  the  gap  between  the  expert  and  the  novice. 

Regarding  the  number  correct  across  the  three  conditions,  the  Elaborated  and  Original  Texts 
produced  fairly  similar  performances,  but  performance  wasn't  significantly  better  than  those  in  the 
Unelaborated  Text  condition.  In  addition,  when  looking  at  the  total  time  subjects  spent  reading  and 
working  on  the  ta  ks,  the  investi-itor  assumed  that  subjects  who  studied  the  22-page  Elaborated  Text  would 
spend  more  time  reading  than  those  who  had  the  Unelaborated  Text  and  the  6-page  Original  Text.  The 
results  show  that  subjects  who  had  the  Elaborated  Text  spent,  on  average,  only  1-1/2  to  2  minutes  more 
time  reading  than  subjects  who  had  the  Unelaborated  Text  and  the  Original  Text.  However,  comparing  the 
combined  times  of  reading  and  working  on  tasks  for  each  group  in  Wendt's  study,  on  average,  subjects  in 
the  Original  Text  group  spent  the  most  total  time  (58.13  min.),  while  subjects  in  the  Unelaborated  Text 
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group  spent  the  least  total  time  (37.22  min.)  and  subjects  in  the  Elaborated  Text  group  spent  the  second 
least  total  time  (41.57  min.).  Wendl  reported  that  as  compared  to  those  who  read  the  other  text  versions, 
those  who  read  the  Elaborated  Text  performed  better  than  the  other  versions  but  did  not  spend  significantly 
more  time  reading  their  text  (26.57  min.  while  28.28  min.  in  the  Original  group)  and  spent  less  time 
(15.00  min.)  working  on  the  application  tasks  (when  20.57  min.  in  Unelaborated  group  and  29.85  min.  in 
the  Original  group).  To  compare  the  learning  efficiency  of  the  performance  among  the  three  groups  in  the 
current  study,  the  data,  calculated  from  the  mean  percentage  of  elements  correct  for  tasks  of  each  subject 
divided  by  the  total  time  he  or  she  spent,  shows  that,  on  average,  the  Original  Text  group  achieved  1.98% 
elements  correct  per  minute,  while  the  Elaborated  Text  group  achieved  1.80%,  and  the  Unelaborated  Text 
group  achieved  1 .83%  elements  correct  per  minute.  The  result,  differs  from  Wendt's,  indicates  that  people 
who  read  the  elaborated  text  did  not  demonstrate  better  learning  efficiency. 

Subject?'  overall  ratings  of  the  Text  on  the  attitude  survey  showed  no  differences  among  the  three 
versions.  However,  subjects  who  read  the  Elaborated  Text  rated  a  positive  score  of  4.20  to  'examples 
helpful'  on  a  6.00  scale,  but  this  was  not  statistically  significant.  This  resu!'.  suggests  that  not  all 
examples  facilitate  procedural  learning.  On  the  other  hand,  subjects  who  read  the  Unelaborated  Text  rated  a 
positive  score  of  4.35  to  'illustrations  helpful'  on  a  6.00  scale,  but  this  was  not  of  higher  statistical 
significance  than  subjects  who  read  the  other  text  versions  (even  though  all  the  three  text  versions  had  the 
same  illustrations).  It  is  interesting  to  note  that  examples  may  not  always  be  necessary  because  when 
subjects  cannot  locate  relevant  examples,  they  tend  to  use  rules  from  the  illustrations.  That  is,  examples 
and  illustrations  of  elaborations  may  be  helpful  in  some  situations,  but  not  all.  Moreover,  the  wide  spread 
in  Attitude  ratings  of  the  three  text  versions  indicate  that  both  Elaborated  and  Unelaborated  Text  versions 
have  room  for  improvement  Since  attitude  ratings  indicated  that  there  were  positive  and  negative  aspects  of 
the  texts,  it  would  be  useful  to  identify  these  features  and  create  new  texts  that  more  adequately  address 
learner  needs. 

Conclusion 

In  conclusion,  the  absence  of  a  significant  difference  in  the  task  performance  across  the  three  text 
versions  suggests  that  not  all  people  require  elaborations  to  develop  a  good  understanding  of  procedural 
learning.  A  possible  reason  for  this  finding  is  that  the  advanced  use  of  logical  connectors  became  too 
complex  for  some  subjects  since  all  groups  had  a  consistently  low  performance  on  Task  4.  Alternatively, 
some  of  the  tasks  might  have  been  so  simple  that  they  did  not  require  elaboration.  This  assertion  is 
supported  by  the  finding  that  subjects  who  read  the  Unelaborated  Text  without  any  examples  did  not 
pcl'orm  worse  than  other  groups. 

The  absence  of  differences  across  the  three  versions  may  also  be  affected  by  the  fact  that  all 
subjects  were  college  students.  Generally  speaking,  younger  subjects  may  need  more  elaborations  than 
adults  because,  in  the  absence  of  elaborations,  adults  may  try  to  find  other  ways  to  solve  their  problems.  In 
addition,  it's  possible  that,  in  general,  adults  may  have  had  more  computer  experience  than  children. 
According  to  the  background  information  provided  by  subjects  for  this  study,  at  least  90%  of  them  had 
some  experience  with  other  computer  software  applications  (which  may  have  used  similar  navigational 
tools).  Obviously,  the  more  prior  computer  experience  individuals  have,  the  easier  it  is  for  them  ;o 
navigate  throughout  a  new  computer  application.  Therefore,  conducting  further  research  with  younger 
subjects  and/or  individuals  with  less  computer  experience  may  produce  different  results  on  the  effect  of  the 
elaborations. 

It  would  also  be  useful  to  conduct  a  study  in  which  additional  versions  of  the  text  were  created  in 
order  to  isolate  specific  types  of  changes.  For  example,  it  would  be  interesting  to  add  a  combination 
version  of  the  use  of  elaborations  with  more  examples  and  the  use  of  a  minimal  approach  with  less 
statements  to  see  what  kinds  of  instructional  text  would  successfully  facilitate  skill  learning.  Another  idea 
for  a  future  study  was  suggested  by  Redish  in  1987.  She  suggested  that  on-line  tutorials  that  display 
instructional  texts  directly  on  the  computer  screen  would  be  a  fascinating  area  for  research  on  a  document 
that  combines  aspects  of  reading  to  learn  and  reading  to  do.  Moreover,  additional  questions  could  be  added 
to  the  Attitude  Survey  and  the  responses  used  to  improve  the  texts. 

It  would  also  be  interesting  to  conduct  a  study  in  which  an  additional  posttest  or  delayed  posttest 
was  administered  to  examine  the  effects  of  elaborations  on  transferring  a  skill.  Additional  factors  (such  as 
age)  should  be  considered  while  designing  a  new  study.  It  would  be  useful  to  determine  which  features  of 
the  extensive  elaboration  approach  and  the  minimalist  approach  work  best  for  different  types  of  users, 
different  types  of  computer  programs,  and  different  types  of  procedural  learning. 
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